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Executive Summary 


The Linking Learning to Careers (LLC) program offered high school students with disabilities an 
individualized and targeted approach to strategically plan for their futures. The Vermont Division of 
Vocational Rehabilitation (DVR) designed LLC to build on its usual services for high school students to 
emphasize unpaid and paid work-based learning experiences (WBLEs) in integrated environments, 
college exploration and coursework opportunities at the Community College of Vermont, team-based 
guidance and support from DVR staff, dedicated support for assistive technology, and additional 
transportation funding. 


DVR was one of five state vocational rehabilitation agencies to receive a five-year grant in 2016 from the 
U.S. Department of Education’s Rehabilitation Services Administration to implement and evaluate 
demonstration projects that offer WBLEs for students with disabilities. To support a rigorous evaluation, 
DVR relied on a randomized controlled trial design. From April 2017 to December 2018, LLC staff 
recruited 803 participants to participate, assigning 413 participants to a treatment group with access to 
enhanced LLC services and 390 participants to a control group that could access usual transition services. 
DVR continued the program through summer 2021. The COVID-19 pandemic could have affected use of 
LLC services and participant outcomes, particularly for those participants that enrolled in LLC in 2018. 
This report presents evidence on LLC’s impacts up to two years after students enrolled in the program. 


Findings 


We estimated the impact of LLC in three domains (services, education, and employment) that represent 
the main consideration of the program’s outcomes, along with supplementary outcomes that further 
support the primary impact findings and could be of interest to policymakers and practitioners. 


Services. LLC had a large impact on service use in the two years after youth enrolled, particularly 
services that LLC intended to offer. These services included those related to WBLEs, assistive 
technology, postsecondary education, and certain preemployment transition services. One-quarter of LLC 
participants (25 percent) had at least two WBLEs, including one paid (a 16 percentage point higher share 
than that of control group members) and more than two-thirds (68 percent) had at least one WBLE (a 41 
percentage point higher share than that of control group members). 


Education. LLC had a large positive impact on enrollment in postsecondary education. The program 
increased participation in postsecondary education by 8 percentage points: 31 percent of LLC participants 
had ever enrolled in postsecondary education programs compared with 22 percent of control group 
members). The impact was largely driven by increased enrollment in the Community College of Vermont, 
a key component of the intervention. LLC did not affect other educational outcomes, such as high school 
completion and career planning knowledge or activities, the rates of which were high among members of 
the treatment and control groups. 


Employment. LLC did not affect employment for all participants within the first 24 months of program 
enrollment. Similar percentages of treatment and control group members (66 percent and 61 percent, 
respectively) had earnings within 24 months of program enrollment. The program did, however, increase 
the likelihood of earnings among later enrollees by 11 percentage points: 66 percent of LLC participants 
who enrolled in the second half of the program had earnings within 24 months of program enrollment 
compared with 55 percent of control group members who did so. 
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Conclusions 


Although the demonstration’s impacts on services and education are consistent with the LLC model and 
findings from the implementation evaluation, the lack of impacts on employment is inconsistent with the 
program’s expectations. Offering high school students WBLEs, along with individualized supports to 
promote their success with those experiences, should have led to positive employment outcomes. Five 
reasons could explain the lack of impacts: 


e Many youth entered the program with work experience. Youth already working in competitive 
integrated employment might not benefit from the additional employment supports LLC offered, 
particularly unpaid WBLEs, as much as youth might without work experience. Focusing on a youth 
population with specific needs and limitations regarding employment might have resulted in 
employment impacts for the program. 


e Most youth had not graduated high school by 24 months after enrollment. Because LLC enrolled 
students in their sophomore and junior years of high school, many had not graduated high school by 
24 months after enrollment, and some of those who had were enrolled in postsecondary education. 
Assessing employment impacts at this point in their transition to young adulthood might be too early; 
the program could generate impacts in these areas in the years after youth leave high school and 
college, particularly if LLC promotes career awareness and options through the services it offered. 


e The COVID-19 pandemic might have affected employment opportunities. Although the 
pandemic constricted the economy, particularly in sectors in which youth might be more likely to find 
jobs, we found little evidence from analyses of quarterly earnings records that this factor affected the 
employment of youth in the treatment and control groups. 


e Variation in program implementation across district offices could have diluted impacts. In the 
implementation evaluation report (Martin et al. 2021) and this report, we document variation in 
program implementation across district offices. Inconsistent implementation of one or more aspects of 
the LLC model could have diluted the program’s impacts. 


e Either the counterfactual for LLC is particularly strong or the LLC intervention is not 
sufficiently different from usual services. DVR’s demonstration offered an opportunity to test the 
LLC model against usual DVR services with a select group of youth, and the estimated impacts 
should be interpreted within that context. The comparison group is not a broad group of youth with 
disabilities without access to transition services. Rather, the comparison group includes a large 
proportion of youth who already had some type of connection to DVR. The LLC intervention might 
not have been sufficiently different from what youth typically accessed through DVR. Impacts might 
have been larger among participants without strong connections to services before enrolling in LLC. 


With LLC and the other four demonstrations it funded, the Rehabilitation Services Administration 
established a foundation to learn more about WBLE models and how they can promote outcomes for 
transition-age youth. The Rehabilitation Services Administration might consider the positive service and 
education impacts as promising indicators of the LLC model, particularly if future analyses find long- 
term impacts on educational attainment and employment. Because of the relatively young age of high 
school students, the Rehabilitation Services Administration and the broader transition community might 
benefit from a fuller understanding of how exposure to WBLEs and other services affect youth’s career 
trajectories into young adulthood and beyond the limited period used for this and the other demonstration 
evaluations. DVR is well positioned to track service, education, and employment impacts beyond the 24- 
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month period used for this evaluation, which can bolster what DVR and the field learn from the LLC 
demonstration. 
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l. Introduction and background: The Linking Learning to Careers 
program and demonstration 


The transition from childhood to adulthood is a critical juncture, particularly for those with disabilities 
(Federal Partners in Transition 2020). During this period, youth develop and acquire the knowledge, 
awareness, and skills necessary to prepare for adulthood, independent living, or pursuit of postsecondary 
education. Compared with their nondisabled peers, a greater share of youth with disabilities come from 
economically disadvantaged backgrounds, have worse health and poorer academic performance, and 
(along with their parents) have lower expectations for education and employment (Lipscomb et al. 2017). 
These difficulties lead to lower rates of educational attainment and adult earnings (Newman et al. 2011). 
Youth with disabilities are more likely to be involved in the juvenile justice system and use federally 
funded Social Security disability programs, which they continue to rely on into adulthood (Newman et al. 
2011; Stapleton and Martin 2017). Entering adulthood reliant on public benefits can discourage rather 
than support employment because program participants must restrict their earnings to remain eligible for 
benefits. 


Despite the availability of transition supports, many transition-age youth with disabilities and their 
families encounter challenges negotiating complex systems of care as they move from youth-based to 
adult services. For example, most disability services provided to youth during secondary school, such as 
individualized education plans and section 504 Plans, are entitlement-based programs, which provide a 
coordinated set of services using a holistic wraparound approach focused on helping students thrive 
academically and functionally. Conversely, adult disability service systems are more fragmented, 
decentralized, and typically eligibility based, requiring youth and their families to apply or reapply for 
services or to visit multiple service providers for needed supports (Wittenburg and Maag 2002; U.S. 
Government Accountability Office 2012). 


The Vermont Division of Vocational Rehabilitation (DVR) designed the Linking Learning to Careers 
(LLC) program with the aim of providing a more individualized and targeted approach to helping students 
gain confidence and strategically plan for their futures. The Rehabilitation Service Administration (RSA) 
funded DVR to implement and evaluate LLC as one of five RSA-sponsored demonstration projects that 
study early work experiences for students with disabilities. This report presents evidence on the impacts 
of LLC up to two years after students enrolled in the program. 


A. VR agencies are critical sources of transition services 


State vocational rehabilitation (VR) agencies, together with secondary schools, can play an important role 
in the transition of youth with disabilities. The VR program is the largest publicly funded program 
offering services to people with disabilities interested in preparing for and engaging in competitive 
employment. VR services can include rehabilitation counseling, assistive technology (AT), job 
accommodations, job search and placement assistance, education and training, and other services and 
support necessary to achieve an employment goal. 


Recently, the roles of VR agencies in serving youth and students with disabilities have changed in 
response to the Workforce Innovation and Opportunity Act (WIOA; Pub. Law. 113-128, 128 Stat. 1425. 
July 22, 2014), which was enacted in 2014. Recognizing challenges faced by this population, 
policymakers passed WIOA to consolidate and strengthen job training programs and increase the role of 
VR in transition. Before WIOA, federal regulations limited VR agencies to serving students eligible for 
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VR services. After WIOA, VR agencies must allocate at least 15 percent of their federal matching grant 
funds to provide preemployment transition services to students with disabilities (RSA 2020). More 
specifically, WIOA allows agencies to offer these services to students who have not yet applied but are 
potentially eligible for VR services. Preemployment transition services include work-based learning 
experiences (WBLEs) and four other required services: job exploration counseling, transition or 
postsecondary education counseling, workplace readiness training, and self-advocacy instruction. 


Of the preemployment transition services available under WIOA, WBLEs might be particularly beneficial 
for youth with disabilities. Evidence suggests that paid work experiences during high school effectively 
increase adult employment (Carter et al. 2012), and initiatives for youth with disabilities that focus on 
education, training, and WBLEs are associated with positive outcomes (Fraker 2014; Mazzotti et al. 
2021). Even an unsuccessful or short-term job can provide a valuable learning experience for youth with 
disabilities (Gerhardt 2006/2007; Cadigan et al. 2006/2007). Work-based programs for youth with 
disabilities also increase the likelihood that youth will acquire higher quality employment outcomes, such 
as jobs with fringe benefits (Shandra and Hogan 2008). 


Recognizing the potential benefits of WBLEs, RSA funded five demonstration projects to highlight early 
work experiences for students with disabilities. Vermont’s DVR designed its LLC program, one of the 
selected demonstrations, to offer holistic, coordinated transition services and multiple individualized 
WBLEs during high school to improve educational and employment outcomes for students with 
disabilities. 


B. Overview of the LLC program 


LLC built on DVR’s usual services for high school students to emphasize unpaid and paid WBLEs in 
integrated environments. LLC also provided college exploration and coursework opportunities at the 
Community College of Vermont (CCV), team-based guidance and support from VR staff, dedicated AT 
support, and additional transportation funding to support participation (Table I.1). Using a random 
assignment evaluation design, LLC offered specific services to 413 participants as part of a treatment 
group, and 390 participants in the control group accessed usual transition services, including those from 
DVR. To facilitate WBLE opportunities and service delivery, DVR funded transition counselors, career 
consultants, and youth employment specialists, some of which only worked LLC caseloads, in each of its 
12 district offices. LLC participants could use other preemployment transition services, along with 
services as VR clients.' More information about the program is available in the LLC implementation 
evaluation report (Martin et al. 2021). DVR continued the program through summer 2021. 


' All participants could access preemployment transition services, but only participants with an open DVR case were 
eligible for other DVR services. 
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Table I.1. Services available for treatment and control group members 


Treatment 
WBLEs 


The LLC goal was to provide each treatment group 
member with three WBLEs during their involvement in the 
program, including one job shadow experience, one unpaid 
internship or volunteer experience, and one employer-paid 
WBLE. Youth could also obtain competitive integrated 
employment. LLC staff used assessment tools, such as the 
O*NET Interest Profiler, to align youth’s interests with job 
opportunities. 


(Oey aii Ke) | 


Control group members were eligible to pursue 
WBLEs as part of preemployment transition services 
and could use online assessment tools such as 
O*NET, but completion of three WBLEs was not a 
goal. 


Preemployment transition services 


Preemployment transition services were available for 
treatment and control group members, but preemployment 
transition services for treatment group members were 
embedded within the LLC framework and provided by the 
LLC team. 


VR services 


All students with disabilities, including those with an 
IEP, section 504 plan, or documented disability, were 
eligible for preemployment transition services. Such 
services help students with disabilities advocate for 
themselves, gain employment experience, and 
prepare for life after high school. 


Services for VR clients were the same for treatment and 
control group members. 


Students determined to be eligible for VR could 
receive a range of services as identified in their 
individualized plan for employment, including career 
and training services, transportation, AT, and financial 
education counseling. 


ed E-Talaliare mxete) by 


LLC staff used optional planning tools developed 
specifically for LLC, including the LLC Career Pathway 
Plan, to gather treatment group members’ short- and long- 
term career goals for their transition after high school. 


Staff who provided services to control group members 
could use other non-LLC planning tools. 


AT services 


LLC staff referred treatment group members to dedicated 
AT specialists as needed. AT services typically began with 
general information sessions to better understand the 
areas in which youth might benefit from AT, after which the 
AT specialists met individually with youth to demonstrate 
products and help them make informed choices. The LLC 
program then loaned or purchased the products for the 
youth, and the specialists provided ongoing follow-up 
support to monitor the youth’s progress. 


VR staff could offer AT support to control group 
members who were within six months of high school 
graduation. The VR program could not, however, 
purchase equipment or devices for control group 
members, and any AT that high schools provided 
remained with the school, not the student, upon 
graduation. Control group members would need to visit 
one of three AT Centers in the state for services. 


Ledoy-) k-y-Lexolalor-lavarcve[ULer-Litey a 


In collaboration with the CCV system, LLC offered several 
enhanced postsecondary education services for treatment 
group members, including campus tours and information 
sessions, specialized academic programs, multiple 
opportunities for dual-enrollment courses, and vouchers to 
cover the tuition for two courses (in addition to the two 
courses available through the state education agency). 
CCV staff worked with treatment group members to better 
understand the linkages between their academics and 
career objectives. 


Control group members could receive vouchers to 
cover the tuition for two college courses worth up to 
four credits each at participating institutions. 
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Treatment OFey al xe) | 

Transportation services 

LLC offered additional funds to support treatment group Control group members with a VR case could obtain 
members’ transportation needs related to education or support for transportation services. Financial support 
work opportunities. Transportation funds were intended to | could be requested from the case services funds or 
be flexible and cover the costs of public transportation VR staff could coordinate with schools to provide 


options, taxis, and a wide range of personal expenses such | transportation. 
as gas, reimbursed mileage, vehicle maintenance or 
purchase, and driver’s permit or license fees. 

AT = assistive technology; CCV = Community College of Vermont; DVR = Division of Vocational Rehabilitation; IEP = 
individualized education plan; LLC = Linking Learning to Careers; VR = vocational rehabilitation; WBLE = work-based 
learning experience. 


To assess the impacts of the program, LLC used an experimental design to randomly assign students to 
receive LLC services or usual services. LLC staff enrolled 803 high school students with disabilities 
across 12 district offices, with 413 students randomized to the treatment group and 390 to the control 
group (Table I.2.). Treatment and control group sizes were similar at each location, with differences of no 
more than five participants between groups. 


_————— Ss 
Table I.2. District offices of LLC participants at enrollment 


District office Treatment (Oey aii Ke) | Difference 


Barre-Montpelier 36 37 -4 
Bennington 38 37 1 
Brattleboro 39 34 5 
Burlington 49 48 0 
Middlebury 17 12 5 
Morrisville 34 33 2 
Newport 35 34 2 
Rutland 34 33 1 
Springfield 29 27 2 
St. Albans 34 31 3 
St. Johnsbury 34 31 3 
White River Junction 34 33 1 


Source: DVR administrative data. 


Notes: Some participants moved over the course of the demonstration. This table reflects participants’ location at 
enrollment. The counts do not include 29 youth that participated in an early pilot of LLC as described in 
Martin et al (2021). 


DVR = Division of Vocational Rehabilitation; LLC = Linking Learning to Careers. 


The implementation evaluation report (Martin et al. 2021) documented that although LLC achieved its 
goal of enrolling 800 students, some LLC staff encountered challenges marketing LLC and encouraging 
students to enroll. Most LLC participants had at least one WBLE and other preemployment transition 
services during their first 18 months in the program, and one in five participants used LLC’s 
postsecondary education, AT, or transportation services. Although LLC participants were satisfied with 
WBLEs, participating in three WBLEs during high school might have been an ambitious goal for the 
program. LLC service uptake varied across DVR district offices, with staff turnover hindering the 
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consistency of service delivery in many district offices. Services also differed by participants’ 
characteristics: participants who were older, had previous employment experience, or enrolled earlier in 
the demonstration were more likely to engage in WBLEs and selected postsecondary education services 
than other participants. 


C. Report organization 


This report describes the processes and findings of the impact evaluation of LLC up to 24 months after 
enrollment. The next chapter describes the evaluation methods, including the data sources and 
descriptions of the analyses. Chapters III to VI present the estimated impacts of LLC on service use, 
education, employment, and expectations for the future, respectively. Chapter VI summarizes key 
findings across chapters and discusses implications for policy and practice. 


The report also contains three appendices with supplemental information. Appendix A provides additional 
detail on data sources and outcome measures as well as the methods for the baseline equivalence 
assessments and the impact analysis. Appendix B highlights the sensitivity analyses and additional 
outcomes not included in the main report. Finally, Appendix C presents the unadjusted treatment and 
control group means for all outcomes. 
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ll. Evaluation methods 


In this chapter, we describe the analytic framework we used to evaluate the impacts of the LLC 
demonstration in Vermont. We include a description of the motivating research questions, outcomes, data 
sources, statistical methods, and subgroups for which we estimated separate impacts. Appendix A 
provides additional methodological details. 


A. Analytic framework 


DVR and Mathematica designed the demonstration to meet the highest standards of the What Works 
Clearinghouse for a rigorous demonstration and evaluation. Key design elements for meeting this level of 
rigor are an experimental design, adequate sample size, low attrition, and comprehensive findings without 
spurious impact estimates (Institute of Education Sciences 2014). 


The demonstration relied on individual random assignment to support the experimental design. From 
April 2017 to December 2018, LLC staff recruited 803 students who were already receiving DVR 
services to participate in the demonstration on a rolling basis.” Because of the recruitment process, the 
impact evaluation assessed the value of LLC services relative to usual DVR transition services rather than 
school-based or no transition services. After parents provided consent, DVR used an algorithm to 
randomly assign half the students to a treatment group and half to a control group.’ All youth in the 
demonstration had access to the usual transition and VR services that DVR offers, if eligible as we 
describe in Chapter I (Table I.1), and youth in the treatment group could also use LLC services. Access to 
LLC services was because of chance, not choice, so any observed differences in outcomes between the 
treatment and control groups is likely attributable to the LLC program. Mathematica’s implementation 
evaluation confirmed that LLC followed random assignment procedures without any contamination that 
could bias the interpretation of results. Both groups had similar demographic characteristics, family 
background, expectations, and health conditions as measured at enrollment (Table II.1), furthering our 
confidence in the experimental design of the impact evaluation. 


Within this experimental design, we used an intent-to-treat framework to estimate LLC’s impacts. With 
an intent-to-treat framework, the evaluation assesses impacts across all members of the treatment group 
regardless of their level of engagement with services. It does not provide estimates of the impact of the 
actual program services on the youth that used them (whether any individual service or all of them as a 
package). Instead, it provides inference of what impact the program would have if replicated elsewhere in 
real world settings, with some youth using more services than others. 


B. Research questions and outcomes 


We estimated impacts on a comprehensive set of outcomes across data sources to answer the following 
research questions: 


e Were treatment group members substantially more likely than control group members to use the 
services intended by the program? 


e Were treatment group members substantially more likely than control group members to attain paid 
competitive employment or enroll in postsecondary education? 


? One student did not enroll until January 2019. 
3 In this report, we use the term parent to refer to parents and guardians. 
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e Compared with the control group, were treatment group members more satisfied with their career 
progress? 


Although we present findings for all outcomes in this report, we selected a single primary outcome in 
each of three domains to determine whether LLC had its intended impact: 


e Services: at least two WBLEs, including one paid, within 24 months of enrollment 
e Education: enrolled in at least one postsecondary education course by December 2020 


e Employment: had at least one quarter of earnings within the eight quarters after enrollment 


Relying on a single primary outcome for a domain addresses issues with multiple comparisons. This issue 
occurs because the probability of detecting spurious impacts increases with the number of outcomes 
tested simultaneously. We selected these three outcomes to indicate LLC’s effectiveness because they (1) 
focus on key LLC components that applied to all treatment group members and (2) can be measured in 
administrative data and are thus less prone to attrition or nonresponse than self-reported or proxy-reported 
survey data. We did not include a primary outcome related to expectations or satisfaction because the 
youth follow-up survey (which asked about expectations and satisfaction) had a low response rate 
(described in the next section). We consider all outcomes other than primary outcomes to be 
supplementary outcomes. We report regression-adjusted treatment and control group differences for 
selected supplementary outcomes in the relevant chapters of the report and for other supplementary 
outcomes in Appendix B. 


We measure most outcomes at or for a duration of about 24 months after a participant enrolled in the 
demonstration. This approach enables us to observe each participant for the same duration despite 
demonstration enrollment occurring over 18 months. The exceptions are education outcomes of 
enrollment in postsecondary education, which we measure through December 2020, because of 
limitations in data access, as we describe in the next section. 


C. Data sources 


We used data from six sources for the impact analysis: (1) LLC enrollment data, (2) a baseline survey, (3) 
DVR administrative data, (4) Vermont unemployment insurance data, (5) the National Student 
Clearinghouse (NSC) data, and (6) 24-month follow-up surveys of youth and parents. 


Two data sources provide information important for the impact evaluation about all participants at the 
time they enrolled in the demonstration. LLC enrollment data identify which participants LLC 
randomized to the treatment and control groups, respectively, their application dates, and the district 
office associated with their applications. The baseline survey collected data to assess baseline 
equivalence across the treatment and control groups and to adjust for background characteristics in the 
multivariate regression models we describe in the next section. These data include demographic and 
disability characteristics, employment history, expectations for the future, and information about their 
parents. Reflecting the rolling enrollment into the demonstration, the program collected these data from 
July 2017 to December 2018. 


Three administrative data sources provide time series information on participants’ services, education, 
and employment from April 2017 to December 2020. We used DVR administrative data from the 
agency’s case management system to measure participants’ use of services during the demonstration. 
These data include all 803 demonstration participants. We used Vermont unemployment insurance 
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data, which contain quarterly earnings, to measure employment and earnings. DVR obtained these data 
from the Vermont Department of Labor for the 715 demonstration participants who provided Social 
Security numbers to DVR. We estimated impacts, however, using only the subset of 638 participants who 
provided valid Social Security numbers at the time of random assignment. Excluding those who did not 
provide a valid number at random assignment avoids the bias of more treatment group members than 
control group members providing Social Security numbers after enrollment because of their LLC 
involvement. Finally, we use DVR’s tabulations from the NSC data to measure enrollment in 
postsecondary education. The NSC collects student-level enrollment and credential information from 84 
percent of all two- and four-year public, private nonprofit, and private for-profit colleges and universities, 
which includes 97 percent of all students enrolled in Title IV degree-granting institutions (Dundar and 
Shapiro 2016). DVR requested data matches for all 803 demonstration participants using names and dates 
of birth and confirmed the accuracy of the matches using corroborating information in case files. Because 
of data-sharing restrictions, DVR could not share the individual records from NSC with Mathematica, so 
it provided tabulations of these data for treatment and control group members. 


The 24-month follow-up surveys of youth and parents provide their respective perspectives on various 
outcomes related to transition. The youth survey asked about services used, recent employment, recent 
education, and expectations for future employment and earnings. The parent survey included questions 
about transition planning and expectations of future employment. Mathematica contacted youth and 
parents to complete the survey roughly 24 months after the youth enrolled in the demonstration, which 
resulted in a data collection period of September 21, 2019, to January 17, 2021. On average, youth and 
parents completed the survey approximately 25.6 months and 25.0 months after enrollment, respectively. 
We obtained responses from 378 youth and 374 parents, with overall response rates of 47 percent for each 
group. The response rates were notably higher among the treatment group youth and parents (52 percent) 
than among the control group youth and parents (42 percent). The low response rates, below the 80 
percent target, might have been partly attributable to the COVID-19 pandemic, which limited our ability 
to interview youth and parents in person. 


D. Estimation methods 


For most outcomes, we used multivariate regression models to estimate LLC’s impact. These models 
enable us to adjust for differences in treatment and control group members’ baseline characteristics. 
Although random assignment resulted in treatment and control groups with similar baseline 
characteristics (Table II.1), controlling for small differences can improve the precision of the estimates. 
This improved precision can sometimes be critical in assessing whether observed differences in outcomes 
are statistically significant when sample sizes or differences are small. For all outcomes, we show the 
unadjusted mean for the control group and the regression-adjusted mean for the treatment group. The 
control group means provide a counterfactual for what the outcomes among treatment group members 
would have been had they not participated in LLC. We report the impact as the difference between the 
unadjusted control group mean and the regression-adjusted treatment group mean, identifying whether 
this impact is statistically significant at three standard p-value thresholds: the 10, 5, and | percent levels. 


An exception to this approach is for the estimates of impacts on postsecondary enrollment outcomes 
measured in the NSC. Because DVR could not share the NSC data with Mathematica, DVR estimated the 
impacts by comparing the unadjusted differences of the treatment and control groups. We tested the 
difference in proportions between the treatment and control groups using the critical value of the normally 
distributed z statistic to assess whether observed differences were statistically significant. 
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Ts 
Table II.1. Characteristics of treatment and control group members at enrollment 


Treatment Difference 


Variable mean Control mean (T —C) 
Number of participants 413 390 
Demographics 


Gender (percentage) 


Male 61.5 59.5 2.0 
Not male 38.5 40.5 -2.0 
Average age (years) 16.7 16.8 0.0 
Race (percentage) 
Black 4.6 5.6 -1.0 
White 93.0 93.6 -0.6 
Other 10.7 9.0 1.7 
Hispanic ethnicity (percentage) 5.3 6.4 -1.1 
Grade level (percentage) 
9th or 10th 45.8 46.7 -0.9 
11th, 12th, missing, and other 54.2 53.3 0.9 
Parent or guardian characteristics 
Education: at least one parent has a college degree 37.0 35.6 1.4 
(percentage) 
Relationship status of responding parent or guardian 
(percentage) 
Single 9.4 11.5 -2.1 
Married or cohabiting 61.5 62.1 -0.6 
Separated, divorced, or widowed 25.4 22.6 2.9 
Any earned income in the past year (percentage) 81.4 82.1 -0.7 


Impairment or condition and health 


Percentage ever identified as having 


ADD or ADHD 45.5 48.2 -2.7 
Autism spectrum disorder 16.2 16.4 -0.2 
Emotional or behavioral disorder or emotional 34.6 39.5 4.9 
disturbance 
Intellectual disability 11.1 10.3 0.9 
Learning disability 49.4 49.7 -0.3 
Speech, hearing or visual impairment 27.4 30.5 -3.2 
Physical impairment or other 15.3 18.7 -3.5 
When disability or condition was first identified 
(percentage) 
Birth 7.3 6.2 1.1 
Before kindergarten 25.4 27.4 -2.0 
Kindergarten to 2nd grade 30.5 28.7 1.8 
3rd to 5th grade 20.6 19.7 0.8 
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Treatment Difference 
Variable mean Control mean (T —C) 


6th to 8th grade 
9th to 12th grade 
Participant's self-reported health (percentage) 


Excellent or very good 57.4 56.4 1.0 
Good 32.9 33.3 -0.4 
Fair or poor 9.4 10.0 -0.6 
Service receipt 
Has an open VR case (percentage) 22.5 24.9 -2.4 
Receives special education services or has an 80.1 83.1 -2.9 
individualized education program (percentage) 
Has a section 504 plan (percentage) 15.3 13.8 1.4 
Receives free or reduced-price school lunch in past 51.8 52.6 -0.7 


12 months (percentage) 


Receives Supplemental Security Income or Social 17.9 18.7 -0.8 
Security Disability Insurance benefits because of a 
disability (percentage) 


Employment 


Worked at a job or business with pay any time in the 53.3 52.8 0.5 
past year (percentage) 
Worked at a job or business without pay any time in 8.7 6.9 1.8 
the past year (percentage) 


Currently working (percentage) 33.7 33.1 0.6 


Hours per week worked at most recent job in the past 
year (percentage) 


Less than 10 hours per week 31.5 28.7 2.8 
10 to 20 hours per week 18.9 17.7 1.2 
21 to 30 hours per week 4.8 5.4 -0.5 
More than 30 hours per week 5.1 5.9 -0.8 
Worked at a school-sponsored work-based learning 13.6 10.3 3.3 


activity in the past year (percentage) 
Expectations 


Expects to attend postsecondary education 67.8 65.4 2.4 
(percentage) 
Expects to be living outside of parent or guardian’s 90.1 89.0 1.1 


home (percentage) 


Definitely or probably expects to be working at a paid 98.1 98.5 -0.4 
job at age 30 (percentage) 


Source: DVR administrative data and LLC baseline survey. 
Note: None of the differences between the treatment and control groups are statistically significant. 


ADD = attention deficit disorder; ADHD = attention-deficit/hyperactivity disorder; DVR = Division of Vocational 
Rehabilitation; LLC = Linking Learning to Careers; VR = vocational rehabilitation. 
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E. Subgroup analyses, impacts across district offices, and sensitivity analyses 


Because some youth might have benefited from LLC more than others, we examined outcomes for 
selected subgroups and across district offices. Although the small number of participants in some of these 
subgroups and each district limits the likelihood of detecting statistically significant differences across 
subgroups, notable differences could still be informative. We selected subgroups that might be relevant to 
the program or policies, might have been affected by gaps in the existing system, and might have been 
more or less engaged with LLC services. We estimated whether the demonstration had a differential 
impact by gender, impairment (has attention-deficit/hyperactivity disorder versus does not have it), grade 
level at enrollment, and income-based eligibility for free or reduced-price school lunch (Tables B.1 and 
B.2). In addition, because a number of factors that could affect impacts varied across DVR districts, such 
as school districts, economic environments, and staff implementation, we separately estimated impacts on 
primary outcomes across district offices (Table B.3). 


We conducted sensitivity analyses to assess the robustness of our estimates (Table B.4). To determine 
whether the subset of participants that completed the 24-month youth survey experienced impacts 
different from those who did not complete the survey, we report primary service and employment 
outcomes on these two subsets of youth. To assess whether the timing of participant enrollment affected 
the impacts of the demonstration, we compared the impacts on the primary service, education, and 
employment outcomes of youth who enrolled earlier (July 2017 to June 2018) in the demonstration with 
those of youth who enrolled later (July 2018 to December 2018). Youth who enrolled later might have 
benefitted from greater staff experience with LLC, but they might have been more directly affected by the 
COVID-19 pandemic and associated economic conditions than earlier enrollees were. Youth who 
enrolled earlier completed most or all of the 24-month study observation period before the pandemic 
began in March 2020. 


Finally, we report unadjusted means for primary outcomes in Appendix C. These show whether the main 
impact estimates are sensitive to the regression adjustments. 
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Services are the means by which LLC expects to influence adult employment outcomes. Understanding 
the services participants actually used provides context for findings in the other domains. Because of 
LLC’s emphasis on individualized planning, needs assessment, and skill development, we expected to 
find that a higher share of treatment group members had WBLEs and other DVR-related services for AT, 
postsecondary education, and transportation. 


In this chapter, we present findings for primary and supplementary outcomes for the service domain. 
These outcomes include pre-employment transition services and other VR services from administrative 
data that were not the focus of the demonstration. We also present selected participant reports of services 
and perspectives from the survey data. Table B.5 presents additional service outcomes. 


As intended, LLC had a large, positive impact on the primary outcome and many supplementary 
outcomes related to DVR service engagement. Just one quarter of treatment group members, however, 
met the primary goal of participating in at least two WBLEs, including one paid, which suggests that 
reaching this goal before high school graduation might be ambitious for sophomores and juniors. 
Alternatively, it could also reflect the fact that more than half of all participants already had worked for 
pay at enrollment (Table II.1); as a result, some might not have seen the value of participating in DVR- 
specific WBLEs such as job shadows. An important caveat is that we can assess only services and 
WBLEs that DVR staff recorded in their administrative data; treatment and control group members could 
have had work experiences that DVR staff did not document. Our estimates of LLC’s service impact 
might therefore be biased upward if DVR staff kept treatment cases open for longer and control group 
members had work experiences that DVR staff did not document. Findings from survey data reveal that 
most control group members also used a number of services outside of those documented by DVR. 
Treatment group members reported service use—particularly the services that LLC emphasized (such as 
education and AT)—at higher rates and had lower reports of having unmet needs than control group 
members did. 


A. Primary outcome: At least two WBLEs, including one paid 


The primary outcome for the service domain is whether a participant had at least two WBLEs, including 
one paid, within 24 months of enrollment. This measure aligns with RSA’s expectations for the 
demonstration but is short of DVR’s goal to facilitate three WBLEs for all treatment group members 
during the demonstration. From DVR’s administrative data, we identified WBLEs as job shadows, other 
unpaid WBLEs such as internships or volunteer experiences, and competitive integrated employment. 
LLC had a large statistically significant impact on this measure of service use. One-quarter of treatment 
group members (25 percent) had at least two WBLEs, including one paid (Table III.1). This rate was 
more than two-and-a-half times that of control group members (9 percent). 
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Table Ill.1. LLC’s impacts on DVR services within 24 months of enrollment 


Treatment (Oey al ine) | 
Outcome mean mean 


Primary outcome 


Had at least two WBLEs, including one paid (percentage) 24.8 9.2 15.6*** (2.6) 
Supplementary outcomes 
Had at least one work experience (job shadow, WBLE, or CIE) 68.4 27.7 40.7*** (3.2) 
(percentage) 
Had at least one job shadow (percentage) 8.9 3.3 5.6*** (1.7) 
Had at least one unpaid WBLE other than job shadow 34.6 11.5 23.1*** (2.9) 
(percentage) 
Had at least one CIE (percentage) 40.2 18.5 21.8*** (3.1) 
Number of WBLEs 1.4 0.5 0.8*** (0.1) 
Used any preemployment transition service other than WBLEs 88.2 70.8 17.5*** (2.8) 
(percentage) 
Job exploration services (percentage) 81.0 62.1 18.9*** (3.1) 
Counseling on postsecondary education opportunities 60.5 36.7 23.9*** (3.4) 
(percentage) 

Workplace readiness training (percentage) 30.0 28.5 1.5 (3.2) 
Self-advocacy training (percentage) 23.4 19.0 4.5 (2.9) 
Has or had an open DVR case (percentage) 94.7 58.0 36.7*** (2.7) 
Had a DVR case that closed with an outcome of CIE 0.0 1.8 -1.8*** (0.7) 

(percentage) 
Used any non-LLC DVR services other than 59.5 25.4 34.1*** (3.2) 
preemployment transition services (percentage) 


Source: DVR administrative data. 


Notes: N= 413 treatment group members and 390 control group members. We regression adjusted treatment 
means and impacts to account for baseline characteristics. Standard errors of the impact estimates are in 
parentheses. 


“r“r"* Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t- 
test. Significance thresholds are not adjusted for multiple comparisons. 


CIE = competitive integrated employment; DVR = Division of Vocational Rehabilitation; LLC = Linking Learning to 
Careers; WBLE = work-based learning experience. 


B. Supplementary outcomes 


The findings for the supplementary outcomes in the services domain support the main finding that LLC 
treatment group members were more likely to receive employment-promoting services than control group 
members. The supplementary outcomes span five categories: other WBLE outcomes, use of 
preemployment transitions services, DVR case status and services (all from DVR administrative data, 
Table III.1), self-reported service use, and satisfaction with services and progress toward goals (from the 
youth follow-up survey, Table III.2). 


Other WBLE outcomes. Reflecting the impact on the primary outcome of two WBLEs, LLC increased 
use of each type of WBLE and the average number of WBLEs. Treatment group members had 
significantly higher rates of engaging in at least one WBLE and each of the three types of WBLEs—job 
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shadows, other unpaid WBLEs, and competitive integrated employment—than control group members 
did. More than two-thirds (68 percent) of treatment group members had at least one WBLE, which is 
more than twice the rate of control group members (28 percent). Job shadows were the least frequently 
used WBLE for treatment and control group members, and higher shares of participants had other unpaid 
WBLEs or competitive integrated employment, particularly among the treatment group. In total, 
treatment group members had an average of 1.4 WBLEs, which was significantly higher than the control 
group’s average of 0.5 WBLEs. 


Use of preemployment transition services. WIOA requires VR agencies to offer students with 
disabilities four categories of preemployment transition services in addition to WBLEs. LLC increased 
use of those other preemployment transition services. Most treatment and control group members used at 
least one other preemployment transition service, and treatment group members were 18 percentage 
points more likely to have used any of these services. Treatment group members engaged in job 
exploration services and counseling on postsecondary education opportunities at significantly higher rates 
than did control group members (by 19 and 24 percentage points, respectively). LLC did not affect 
participation in workplace readiness or self-advocacy training, which had the lowest uptake rates in both 
experimental groups. 


DVR case status and services. As intended, treatment group members applied and were found eligible 
for DVR services at significantly higher rates than their control group counterparts. Although 58 percent 
of control group members had an open DVR case at some point within 24 months of LLC enrollment, 
nearly all (95 percent) treatment group members attained this milestone. Few participants in either group 
exited VR services during this time period; 2 percent of control group members had a DVR case that 
closed because they attained a competitive integrated employment position compared to none of treatment 
group members. This pattern is consistent with LLC’s intent to keep treatment group members connected 
with DVR throughout the demonstration as a way of establishing a bridge between youth and adult 
services. Finally, more than half (60 percent) treatment group members used DVR services other than 
preemployment transition services and LLC-specific services during the two years after enrolling in the 
demonstration. Just over a quarter of control group members did the same. 


Self-reported services used. LLC affected youths’ self-reported service use in ways that were consistent 
with the project’s logic model. The large majority of treatment and control group members reported use 
of any services (99 percent versus 93 percent, respectively; Table III.2). Nonetheless, the 6 percentage- 
point difference between the groups is statistically significant. Treatment group members accessed 5 of 
the 13 services queried in the survey at significantly higher rates than their control group counterparts: 
information and referral services, services to help with life skills, services to help learn about or get into a 
school or training program, job skills training, and AT services. In addition, half or more of both 
treatment and control group youth reported using the following services: information and referral, life 
skills, activities to learn about what jobs match one’s skills and interests, finding or applying for jobs, and 
how to save or manage money. About 7 percent of control group members reported that they did not 
access any needed help or services in preparing for work or school, a significantly higher share (by 6 
percentage points) than for treatment group members. Despite treatment group members using AT 
services at a higher proportion than control group members did, the two experimental groups did not 
differ in the proportions using AT tools often or always. 


Satisfaction with services and progress toward goals. LLC did not affect participants’ overall 
satisfaction with services. Most enrollees (85 percent of treatment group members and 87 percent of 
control group members) were very or somewhat satisfied with the services they used to advance in school 
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or prepare for a job after school, and roughly three-quarters of both groups indicated that they were very 
or somewhat successful in reaching their goals. Treatment group members, however, were significantly 
more likely to report being satisfied with the AT tools they used (Table B.5). 


Se 
Table III.2. Participant reports of service use and satisfaction 


Treatment 
mean Control mean | Difference 
Number of survey respondents 214 164 
Reported using any services in the past year (percentage) 98.7 92.9 5.7** (2.9) 
Service type (percentage) 
Information and referral 76.1 66.1 10.0* (5.2) 
Life skills 68.9 59.8 9.1* (5.5) 
Finding or applying for a job 62.1 56.7 5.4 (5.6) 
Activities to learn about what jobs match skills and 60.6 54.0 6.6 (5.6) 
interests 
Learning about or getting into a school or training program 59.1 42.3 16.7*** (5.5) 
How to save and manage money 52.9 51.8 1.1 (5.8) 
Self-advocacy training 41.4 45.0 -3.6 (5.5) 
Job skills training 35.9 26.8 9.0* (5.4) 
Transportation to or from a workplace activity 34.8 39.6 -4.8 (5.4) 
Assistive technology 32.4 19.3 13.1*** (4.9) 
Benefits counseling 20.2 19.0 1.2 (4.5) 
Help while working at a job 19.1 18.2 0.9 (4.2) 
Other services to help them work, go to school, or help 9.0 6.9 2.1 (3.4) 
their family in other ways 
In the past year, had unmet needs related to preparation for 1.3 7.4 -5.7** (2.9) 
work or school (percentage) 
Very or somewhat satisfied with the services used to help 84.6 87.3 -2.7 (4.1) 
advance in school or prepare for a job after school 
(percentage) 
Very or somewhat successful in reaching goals (percentage) 76.4 74.6 1.8 (4.9) 
Always or often use assistive technology tools (percentage) 26.6 21.6 5.0 (4.4) 


Source: LLC youth follow-up survey. 


Notes: N= 214 treatment group members and 164 control group members who responded to the survey. Sample 
sizes vary slightly across survey responses. We regression adjusted treatment means and differences to 
account for baseline characteristics. Standard errors of differences are in parentheses. 


“r“r"* Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 
Significance thresholds are not adjusted for multiple comparisons. 


LLC = Linking Learning to Careers. 


C. Subgroup and district office differences 


LLC might have had a greater impact on some enrollees because the services they otherwise would have 
accessed were less robust, because the participants were more engaged with LLC, or because of variation 
in implementation across DVR districts or in the school and community environment. We did not find 
statistically significant differential impacts across the five subgroups we examined: gender, disability 
type, grade at enrollment, early versus later enrollment, and eligibility for free or reduced-price school 
lunch (Table B.1). We did find that the impact of LLC on the primary service outcome varied 
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substantially across district offices, with larger-than-average impacts in some district offices and no 
impacts in other district offices (Table B.3). 
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IV. LLC’s impacts on education outcomes 


The LLC program expected that enhanced services offered to the LLC treatment group would positively 
impact their education outcomes. These outcomes included higher rates of participation in postsecondary 
education and training opportunities as well as greater clarity on how to attain a job compared with the 
control group. In the longer term, a positive impact of LLC on education outcomes could also lead to 
better employment outcomes. 


We present findings on primary and supplementary outcomes in the education domain. The primary 
outcome in this domain is enrollment in postsecondary education at CCV and other postsecondary 
institutions as measured in NSC administrative data. We also present selected participant reports of 
education and their perspectives on career planning from the youth follow-up survey. Table B.6 presents 
additional education outcomes. 


LLC led to a large and statistically significant increase in postsecondary education enrollment. LLC was 
not associated with differences in other educational outcomes, such as high school completion and career 
planning knowledge or activities, the rates of which were relatively high among members of the treatment 
and control groups. Thus, there is no evidence that these outcomes drove the impact on postsecondary 
education enrollment. LLC’s impact on the use of education services, however, might have contributed to 
the positive impact (shown in Chapter III). Substantially more treatment group members than control 
group members obtained counseling on postsecondary education opportunities (Table II.1) and reported 
using services to learn about school and training opportunities (Table III.2). These outcomes on 
postsecondary education services and enrollment reflect LLC’s focus on postsecondary education as 
described in the implementation report (Martin et al. 2021). 


A. Primary outcome: Ever enrolled in postsecondary education 


The primary outcome for the education domain is whether a participant ever enrolled in postsecondary 
education. From NSC administrative data, we measured this outcome by identifying enrollment at 
technical training schools, two-year community colleges, and four-year colleges through December 2020. 
LLC increased enrollment in postsecondary education. More than 30 percent of treatment group members 
attended postsecondary education institutions, a rate that is 8 percentage points higher than that of control 
group members (22 percent) (Table IV.1). 


LLC encouraged treatment group members to engage in postsecondary education services such as campus 
tours and information services, dual enrollment courses, and specialized contract courses (Martin et al. 
2021). LLC offered most of these services through a partnership with CCV that connected treatment 
group members to postsecondary education opportunities. Our understanding is that enrollment in these 
introductory courses would not be documented in the NSC data, so the observed impacts on 


postsecondary education in the NSC data likely do not include courses that were part of the LLC program. 


Nonetheless, the NSC data might include some courses that youth took as part of the LLC program. 
Although we cannot attribute any single component of LLC to the increase in postsecondary education 
enrollment, LLC’s partnership with CCV might have been instrumental in generating this impact 
enrollment. 
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B. Supplementary outcomes 


The supplementary outcomes we assess in the education domain span two categories: specific types of 
school enrollment (from the NSC data, Table IV.1) and academic and career planning (from the youth 
follow-up survey, Table IV.2). 


_—_——_—SeeS SSS Se 
Table IV.1. LLC impacts on postsecondary education through December 2020. 


Treatment Control 
Outcome mean mean 


Primary outcome 


Ever enrolled in postsecondary education (percentage) 30.6 22.4 8.2*** (3.1) 
Enrolled in courses at CCV (percentage) 22.3 14.7 7.6*** (2.7) 
Enrolled at other postsecondary education institutions (percentage) 13.6 11.3 2.3 (2.3) 


Source: National Student Clearinghouse data. 


Notes: N= 413 treatment group members and 390 control group members. Standard errors of the impact 
estimates are in parentheses. 


“r“"* — Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t- 
test. Significance thresholds are not adjusted for multiple comparisons. 


CCV = Community College of Vermont 


Other school enrollment outcomes. LLC increased enrollment in CCV by about 8 percentage points, but 
it had no impact on other types of school enrollment. About 22 percent of treatment group members and 
15 percent of control group members enrolled in at least one course at CCV through December 2020 
(Table IV.1). With nearly 100 percent of treatment and control group members successfully completing or 
continuing with secondary school, we find no evidence that the LLC program had an impact on 
completion or continued enrollment in high school (Table IV.2). Although the share of participants 
enrolled in postsecondary education in each group at the time of the survey interview was not 
significantly different, the size of the estimate based on the follow-up survey is consistent with the size of 
the NSC data estimate. At the time of the survey, 31 percent of the treatment group and 23 percent of the 
control group reporting being enrolled in postsecondary education (Table IV.2). 


Academic and career planning. The LLC program did not have an impact on academic and career 
planning or knowledge. Most LLC participants agreed or strongly agreed with each of the five academic 
and career planning statements, reflecting that they had the necessary tools to plan for a future career of 
their choice (Table IV.2). About three-quarters of participants agreed or strongly agreed with three 
measures of career planning knowledge (knowing where to get more information about jobs, what jobs 
were a good fit for them, and the requisite skill for these jobs). Slightly fewer reported having engaged in 
each of two specific career planning activities (taking steps to learn about different jobs and having a plan 
outlining the steps they have to take to get the job they want). The responses did not vary significantly 
across the experimental groups. 
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Table IV.2. Student reports and perspectives on education outcomes, 24 months after enrollment 


Treatment (Oey aii Ke) | 
mean mean Difference 


Completed or currently enrolled in secondary school (percentage) 97.9 98.8 -0.8 (1.1) 
Enrolled in postsecondary education at the time of the survey 30.7 23.4 7.3 (5.1) 
(percentage) 
Agree or strongly agree (percentage): 
| know what types of jobs are best for me 83.8 84.5 -0.8 (4.0) 
| know what skills and training | need for the job(s) | want 76.8 78.7 -2.0 (4.7) 
| Know where to go online to learn about different types of jobs 75.0 80.0 -5.0 (5.0) 
and careers as well as the education and training needed for 
them 
| have taken steps, such as visited businesses or taken career 70.7 65.2 5.5 (5.2) 
surveys, to learn about different kinds of jobs 
| have a plan outlining the steps that need to be taken to obtain 65.9 69.1 -3.2 (5.6) 


the job(s) | want 


Source: LLC youth follow-up survey. 


Notes: N= 214 treatment group members and 164 control group members that responded to the survey. Samples 
sizes vary slightly across survey responses. We regression adjusted treatment means and differences to 
account for baseline characteristics. Standard errors of differences are in parentheses. 


*r*** — Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 
Significance thresholds are not adjusted for multiple comparisons. 


LLC = Linking Learning to Careers. 


C. Subgroup differences 


The estimated impact of LLC on postsecondary education enrollment was positive and significant among 
LLC participants who enrolled in the demonstration earlier (July 2017 to June 2018) but not for those 
who enrolled later (July 2018 to December 2018) (Table B.4). This pattern is not surprising because 
earlier enrollees were older, and we observed their educational outcomes over a longer duration (an 
additional year). The COVID-19 pandemic might also have disproportionately dampened or delayed 
postsecondary education enrollment among participants who enrolled later. Because of data limitations, 
we could not analyze other subgroup differences for the primary outcome of the education domain. 
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V. LLC’s impacts on employment outcomes 


The driving motivation for the LLC program was to improve employment outcomes for youth and young 
adults with disabilities. Youth employment is a strong predictor of adult employment, so it could be 
considered an indicator of whether transition-age youth will go on to have a career later in their lives. As 
we describe in Chapter III, one LLC goal was for treatment group members to have at least one paid work 
experience during the demonstration. 


In this chapter, we describe LLC’s impact on primary and supplementary employment outcomes. We 
used Vermont unemployment insurance data from the Vermont Department of Labor to measure 
employment and earnings in the eight calendar quarters after the quarter in which each participant 
enrolled in the demonstration. We used data from the youth follow-up survey for participants’ 
perspectives on their employment. Table B.7 presents additional employment outcomes. 


LLC did not have a significant impact on any of the primary or supplementary employment outcomes, 
though it did have a positive impact on employment for one of the examined subgroups. Most participants 
had at least one quarter of earnings within the eight quarters after enrollment. This finding supports 
observations at baseline, in which more than half of participants reported working in the previous year 
and one-third had jobs at the time of their enrollment in the demonstration. Although the COVID-19 
pandemic could have affected the LLC program’s impact on employment, we have no evidence that it did 
so in ways that differed across the experimental groups. Both treatment and control group members had 
lower rates of documented earnings in the second quarter of 2020, but those rates rebounded the 
following quarter (Table B.7). For those who enrolled in the demonstration in the later part of the 
enrollment period (from April 2018 to December 2018), the estimated impact on the primary employment 
outcome was 16 percentage points higher than for those who enrolled in the first part of the program. 
Long-term employment impacts of LLC could be measured in the future as participants embark on 
careers, especially given the impact of LLC on postsecondary education enrollment (Chapter IV). 


A. Primary outcome: Had at least one quarter of earnings 


The LLC program did not have an impact on earnings. The primary outcome for the employment domain 
is whether a participant had at least one quarter of earnings within the eight quarters after the quarter in 
which they enrolled in the demonstration. Nearly two-thirds of all LLC participants had such earnings in 
this period regardless of experimental group (Table V.1). 
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Table V.1. Impacts of LLC on employment outcomes during eight quarters after enrollment quarter 


Treatment | Control 
mean mean 


Number of participants who provided Social Security numbers 


Primary outcome 


Had at least one quarter of earnings (percentage) 65.6 61.0 4.7 (3.8) 


Supplementary outcomes 


Any earnings (percentage) 


First four quarters after enrollment 49.0 46.5 2.6 (4.0) 
Second four quarters after enrollment 59.5 56.9 2.6 (3.9) 
Average weekly earnings ($2021) 60.2 65.1 -5.0 (6.9) 
First four quarters after enrollment 40.6 46.0 -5.4 (5.7) 
Second four quarters after enrollment 79.7 84.2 -4.5 (9.3) 


Source: DVR administrative data and Vermont unemployment insurance data. 


Notes: N= 339 treatment group members and 299 control group members that provided Social Security numbers 
at enrollment. We regression adjusted treatment means and differences to account for baseline 
characteristics. Standard errors of the impact estimates are in parentheses. 


*/*/*** Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 
Significance thresholds are not adjusted for multiple comparisons. 


DVR = Division of Vocational Rehabilitation; LLC = Linking Learning to Careers. 


B. Supplementary outcomes 


Our analyses across three categories of supplementary outcomes related to employment (employment and 
earings, engagement, and job satisfaction) provide further evidence for LLC’s lack of an impact on 
employment. 


Employment and earnings. LLC did not have an impact on any of the supplementary outcomes related 
to employment and earnings as measured in administrative or survey data. From the administrative data, 
we measured the likelihood of having any earnings separately over the first four and second four quarters 
as well as average weekly earnings in aggregate and separately for the first four and second four quarters 
after the quarter of enrollment (Table V.1). Despite the lack of impacts, earnings rates and average 
weekly earnings increased during the second four quarter period from the first four quarter period across 
both experimental groups. A larger share of participants reported being employed in the survey than we 
observed with earnings in the administrative data (Table V.2). About 80 percent of participants reported 
being employed during the year before the follow-up survey compared with the roughly 60 percent rate 
we observed over roughly the same period in the administrative data. This difference likely reflects paid 
and unpaid work experiences reported in the survey by participants. Similar proportions of treatment and 
control group youth (about half) reported working at the time of the survey. 


Productive engagement. Because of the age of participants, many might enroll in postsecondary 
education, as we describe in Chapter IV, so they might not launch their careers until after college 
completion. As a result, a measure that combines employment and college enrollment can be a useful 
measure of engagement in productive pursuits. Treatment and control group participants were equally 
likely to be engaged in productive pursuits at the time of the survey; nearly two-thirds of both groups 
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reported working or being enrolled in postsecondary education 24 months after program enrollment 
(Table V.2). 


Job satisfaction. LLC had no impact on participants’ satisfaction with their most recent job. Nearly all 
participants who had worked in the year before the follow-up survey reported that they were somewhat or 
very satisfied at their most recent job (Table V.2). This finding suggests that treatment and control group 
members were able to find and obtain jobs that they enjoyed. 


| 
Table V.2. Participant reports and perspectives on employment (percentage) 


Treatment Control 
mean mean Difference 


Employed in last year (percentage) 79.2 83.8 -4.6 (4.0) 
Employed at the time of the survey (percentage) 50.0 53.2 -3.2 (5.7) 
Working or enrolled in postsecondary education at the time of survey 63.5 65.4 -1.9 (5.3) 
(percentage) 

Somewhat or very satisfied at most recent job (among those with a 90.6 85.4 5.3 (4.3) 


job) (percentage) 


Source: LLC youth follow-up survey. 


Notes: N= 214 treatment group members and 164 control group members that responded to the survey. Samples 
sizes vary slightly across survey responses. We regression adjusted treatment means and differences to 
account for baseline characteristics. Standard errors appear in parentheses. 


**["** Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 
Significance thresholds are not adjusted for multiple comparisons. 


C. Subgroup and district office differences 


Although LLC did not have a significant impact on employment in aggregate, it did have a significant 
positive impact on employment for one subgroup of participants. Two-thirds (66 percent) of treatment 
group members who enrolled later in the demonstration had at least one quarter of earnings, a rate that 
was |1 percentage points higher than that of control group members (55 percent). The estimated impact 
on employment was 16 percentage points higher among participants who enrolled in the demonstration in 
the later part of the enrollment period (from April 2018 to December 2018) relative to the estimated 
impact for those who enrolled earlier (Table B.2). We did not observe any differential impacts for the 
other four subgroups (disability type, gender, grade at enrollment, and eligibility for free or reduced-price 
school lunch). 


The finding of no impacts on earlier participants and impacts on later participants could be because of 
three reasons: (1) the program was better implemented over time, (2) the LLC program provided an 
employment buffer against the pandemic recession, or (3) early and late participants differed in important 
ways that could affect how they benefitted from LLC. Although differences between treatment and 
control groups through 2020 are not large enough to draw conclusions about whether the pandemic 
affected LLC’s impact on employment, findings from the implementation report provide some evidence 
for the first and third explanations (Martin et al. 2021). Early and late enrollees are similar across most 
baseline characteristics, though a substantially higher share of early enrollees had prior engagement with 
DVR before LLC enrollment (that is, they had an open VR case or had received preemployment transition 
services). Late enrollees included a higher share of students without such DVR engagement. This 
difference resulted from expanding recruitment to high schools (Chapter 1) and provides suggestive 
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evidence that youth not already engaged with DVR could benefit more from LLC than youth already 
engaged with DVR. 


We also found that the treatment and control group rates of employment varied substantially across 
district offices, with a positive and statistically significant impact in one district office but not others 
(Table B.3). 
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Vi. LLC’s impacts on expectations 


To build confidence about their futures, LLC offered youth opportunities to explore their interests and 
enhance their skills through hands-on work experiences and postsecondary education services. With a 
successful intervention, treatment group members would develop a better understanding of their interests 
and skills through these experiences, which could improve youth’s planning for their futures. Moreover, 
though youth’s participation in the intervention and engagement with enhanced services, parents might 
experience change in their expectations for those youth. 


This chapter highlights the expectations of LLC participants and their parents 24 months after enrollment. 
All results in this chapter are considered supplementary outcomes because although changes in 
expectations help contextualize findings on other outcomes, they were not part of the program’s logic 
model. Table B.8 presents additional expectation outcomes. 


LLC increased youth’s expectations for earning a college degree, but it did not have a detectable effect on 
other expectations of treatment group youth and parents about the future. The impact on college 
expectations is consistent with LLC’s focus on educational opportunities. One factor that might have 
limited the ability of LLC to influence other expectations is that at enrollment, most youth (98 percent) 
reported that they expected to work at age 30, and their parents had similar expectations for them (97 
percent) (Martin et al. 2021). These high expectations at enrollment and 24 months later contrast with 
published research on expectations of students participating in special education (Lipscomb et al. 2017). 
This discrepancy suggests that the youth and families who agreed to participate in LLC were especially 
focused or interested in employment and postsecondary education options, more so than the broader 
group of youth with individualized education plans or section 504 plans. 


A. Supplementary outcomes: Youth and parent expectations 


LLC positively affected expectations about college education but not about work or financial support. 
Almost all participants (95 percent) expected they would definitely or probably work for pay at age 30, 
regardless of experimental group, and nearly three-quarters expected to earn enough to support 
themselves without financial help from family or government benefit programs. Most parents (91 percent 
of treatment and 92 percent of control) also believed their children would work for pay at age 30. These 
high expectations did not vary between treatment and control groups, providing no evidence that LLC 
affected youth or parent expectations for employment. LLC did increase expectations about educational 
attainment. About 46 percent of treatment group members expected to earn a college degree, a rate that 
was 9 percentage points higher than the control group rate of 37 percent (Table VI.1). 
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Table VI.1. Youth and parent reports on expectations 


Treatment 
lO] Korey nay=) mean Control mean Difference 


Youth expectations 


Number of youth respondents 214 163 

Definitely or probably expects to work for pay at age 30 95.0 95.2 -0.2 (2.4) 
(percentage) 

Definitely or probably expects to earn enough to support 70.8 74.9 -4.1 (4.9) 


themselves without financial help from family or 
government benefit programs (percentage) 


Expects to earn a college degree (percentage) 45.8 37.0 8.8* (5.0) 
Number of parent respondents 213 160 
Expects youth to definitely or probably work for pay at age 91.2 91.8 -0.6 (3.0) 


30 (percentage) 
Expects youth to earn a college degree (percentage) 54.1 52.8 1.4 (5.2) 


Source: LLC youth and parent follow-up surveys. 


Notes: Samples sizes vary slightly across survey responses. We regression adjusted treatment means and 
differences to account for baseline characteristics. Standard errors of differences are in parentheses. 


*/*/*** Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 
Significance thresholds are not adjusted for multiple comparisons. 


LLC = Linking Learning to Careers. 
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This report presents the impacts of LLC on important intended outcomes about two years after 
participants enrolled in the program. LLC offered high school students with disabilities a package of 
services to facilitate their transitions from school to young adulthood. These services emphasized paid 
and unpaid WBLEs along with college exploration and coursework opportunities at CCV, team-based 
guidance and support from VR staff, dedicated AT support, and transportation funding. Participants also 
had access to preemployment transition services and usual supports from DVR and other providers that 
were available to all youth with disabilities in Vermont. LLC implemented the program throughout the 
state, enrolling 413 participants into the treatment group and 390 in the control group. The 
implementation report (Martin et al. 2021) identified the essential services available to LLC participants 
during the first 18 months of their participation as (1) a dedicated staff member (the career consultant) 
with a small caseload, (2) a team of other staff who worked with the career consultant to address 
transition needs, (3) preemployment transition services, and (4) WBLEs. Because of the randomization 
process, in which participants either had access to LLC services or usual services, the impact estimates 
reflect the effects of the program rather than the effects of other factors that might have influenced 
outcomes. In this chapter, we summarize the estimated impacts, contextualize the findings, and discuss 
program and policy implications for state VR agencies and RSA. 


A. Summary of impacts 
LLC had the following impacts on participants within 24 months of their enrollment into the program: 


e Services. LLC had a large impact on services in the two years after youth enrolled, particularly 
services that LLC intended to offer. These services included those related to WBLEs, AT, and 
postsecondary education as well as certain preemployment transition services. Nearly 70 percent of 
the treatment group had at least one WBLE recorded in DVR’s administrative data, though only one- 
quarter (25 percent) met the primary outcome measure of having two WBLEs, including one paid. 


e Education. LLC increased enrollment in postsecondary education institutions and, specifically, in 
CCV. This higher enrollment corresponds with LLC’s positive impact on the use of postsecondary 
education services. LLC also increased expectations of attaining a college degree for youth. 


e Employment. Overall, LLC did not affect employment outcomes. Similar percentages of treatment 
and control group members had earnings or paid work experiences within 24 months of program 
enrollment. The program did, however, increase the likelihood of earnings among later enrollees. 


B. Contextualizing the findings 


The demonstration’s impacts on services and education are consistent with findings from the 
implementation evaluation and follow from the context of the DVR services that participants could 
access. LLC offered distinct services for WBLEs, postsecondary education, AT, and transportation as part 
of its program—services that youth in the control group could not access—and had positive impacts on 
many of these measures. The implementation report showed that about one-fifth of treatment group youth 
used many of these program-specific services. Although not all youth required these services, a certain 
proportion could have benefited, and LLC’s offer of these services might have filled needed gaps, such as 
with schools playing a limited role in offering AT for employment-related needs. Youth’s responses to the 
follow-up survey confirm the impact of LLC on the program’s delivery of these services. The impacts on 
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service take-up were consistent with the program’s logic model. LLC’s relationship with CCV in 
particular likely played an important role in connecting youth to that postsecondary education institution. 


The lack of consistent impacts on employment is inconsistent with the program’s expectations. Offering 
high school students WBLEs, along with individualized supports to promote youth’s success with those 
experiences, should have led to positive employment outcomes. We found no impacts, however, using 
administrative or survey data for all enrollees, though we did find impacts for those who enrolled in the 
later part of the program. Five reasons could explain the lack of consistent impacts. 


e Many youth entered the program with work experience. Youth already working in competitive 
integrated employment might not benefit from the additional employment supports LLC offered, 
particularly unpaid WBLEs. The fact that more than half of treatment and control group members had 
already worked when they enrolled in the program suggests that many already had the skills and 
abilities to procure employment on their own or with naturally occurring supports. Focusing on a 
youth population with specific needs or limitations around employment might have resulted in more 
consistent employment impacts for LLC. 


e Most youth had not graduated high school by 24 months after enrollment. Almost two-thirds of 
youth were still in high school at the time we measured the employment outcomes 24 months after 
they enrolled in the program. The remainder might have graduated or otherwise left high school 
recently before we conducted our evaluation activities. Assessing education and employment impacts 
at this point in their transition to young adulthood might be too early; the program could generate 
impacts in these areas in the years after youth leave high school, particularly if LLC promotes career 
awareness and options through the services it offered. 


e The COVID-19 pandemic might have affected employment opportunities. Although the 
pandemic constricted the economy, particularly in sectors in which youth might be more likely to find 
jobs, we found little evidence from our analyses that this factor affected the employment of youth in 
the treatment and control groups. 


e Variation in program implementation across district offices could have diluted impacts. In the 
implementation evaluation report (Martin et al. 2021) and this report, we document variation in 
program implementation across district offices. Inconsistent implementation of one or more aspects of 
the LLC model could dilute the program’s impacts. We did not find strong evidence, however, that 
better implementation, in terms of the district-level impact for two or more WBLEs, consistently 
resulted in larger impacts for employment. 


e Either the counterfactual for LLC is particularly strong or the LLC intervention is not 
sufficiently different from usual services. DVR’s demonstration offered an opportunity to test the 
LLC model against usual DVR services with a select group of youth, and the estimated impacts 
should be interpreted within that context. The comparison group is not a broad group of youth with 
disabilities without access to transition services. Rather, the comparison group includes a large 
proportion of youth who already had some type of connection to DVR. DVR staff conducted initial 
outreach for LLC enrollment to high school students using DVR services, whether eligible or 
presumed eligible for VR, though they later expanded the pool to high school students with 
individualized education plans or section 504 plans. Those youth who had some engagement with 
DVR presumably would be well positioned to continue to use those services throughout and beyond 
high school. Alternatively, the LLC intervention—which built on usual VR services by emphasizing 
connections to WBLEs, expanded AT and postsecondary education services, and more personalized 
staff connections—might not have been sufficiently different from what youth typically accessed 
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through DVR. The services documented by administrative and survey data show that many control 
group members had access to and used an array of transition-related services, including those offered 
by DVR. Impacts might have been larger with a comparison group without such strong connections to 
services. Large, significant impacts on employment of participants who enrolled later in the 
demonstration (a group that had a smaller share of youth that were already connected to DVR) is 
consistent with this explanation. 


DVR is well positioned to track service, education, and employment impacts beyond the 24-month period 
used for this evaluation. With its administrative data on services, it can follow youth’s pathways through 
the DVR system, including identifying which youth continue to use services for education and 
employment. With regular analyses of state wage records, DVR can assess whether employment impacts 
emerge as youth leave high school and postsecondary education. And with data from the NSC, it can 
observe whether the early impacts with postsecondary education enrollment lead to postsecondary 
education completion and identify the types of credentials youth eventually obtain. Such information can 
bolster what DVR and the field learn from the LLC demonstration. 


C. Program and policy implications for state VR agencies 


Administrators from other state VR agencies might be interested in replicating LLC or a similar model in 
their states. If so, they might consider the following questions. 


Should other state VR agencies implement LLC? LLC succeeded in delivering many aspects of its 
intended services and with increasing postsecondary education. These positive impacts, on their own, 
might be sufficient evidence for other agencies to develop a program using LLC’s model. The program 
did not affect employment, however, during the initial 24 months after program enrollment. The positive 
impacts on postsecondary education could result in eventual educational attainment and employment 
impacts as youth enter young adulthood, but administrators might want additional evidence on the 
model’s effectiveness. 


To what extent can LLC be replicated in other states? On the surface, the LLC model could be 
replicated elsewhere. The core aspects of the model (that is, using a team-based approach with dedicated 
staff to offer paid and unpaid WBLEs, postsecondary education exploration and coursework 
opportunities, dedicated AT support, and transportation funding, all of which build on existing VR 
services) are within the capabilities of most VR agencies, as many have pivoted to offer more transition 
services since the passage of WIOA. Replicating the model, however, might require increased transition- 
related staffing and strong community partnerships, particularly with AT providers and one or more 
postsecondary education institutions. 


If replicated, could administrators expect similar success? The answer is maybe. The youth who 
enrolled in LLC have characteristics that differ from the general population of special education students 
nationwide, which might affect the ability for an LLC-like program to have similar results. For example, 
the share of transition-age youth with a disability is higher in Vermont than in the United States overall, 
which could indicate higher prevalence or higher rates of diagnosis among students with less significant 
disabilities. The youth who enrolled in the demonstration were largely White, reflecting the broader 
population of Vermont. Most enrollees worked before they signed up for the program, and they and their 
parents had high expectations for youth’s work, education, and independent living at baseline. The 
students who enrolled in LLC could therefore differ from typical students with disabilities receiving 
services from VR agencies. 
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D. Program and policy implications for RSA 


With LLC and the other four demonstrations it funded, RSA established a foundation on which it could 
learn more about WBLE models and how they can promote outcomes for transition-age youth. RSA 
might consider the positive service and education impacts as promising indicators of the LLC model, 
particularly if future analyses find long-term impacts on educational attainment and employment. Because 
of the relatively young age of high school students, RSA and the broader transition community might 
benefit from a fuller understanding of how exposure to WBLEs and other services affect youth’s career 
trajectories into young adulthood and beyond the limited period used for this and the other demonstration 
evaluations. 


RSA could use the information learned from LLC and the other four demonstrations on service delivery 
and impacts in two ways. First, it could consider technical assistance options for state VR agencies as the 
agencies continue to offer transition services and develop collaborations around transition, particularly 
with secondary and postsecondary education institutions and community rehabilitation providers. This 
information might be particularly salient for RSA-funded technical assistance centers. Second, RSA could 
leverage information collected from youth who participated in the demonstrations to track the long-term 
outcomes that result from youth’s exposure to WBLEs, other preemployment transition services, and 
other VR agency supports offered as part of the demonstrations. Such information could inform what we 
know about the long-term effects of these services on the career trajectories of youth with disabilities. 
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Appendix A Impact Evaluation Methods 


A. Introduction 


In this appendix, we provide more information on the methods we used to estimate the impacts of the 
Linking Learning to Careers (LLC) demonstration in Vermont. The appendix serves as a supplement to 
Chapter II, which describes key components of the analytic framework and methods. 


B. Testing baseline equivalence between experimental groups 


As we describe in Chapter II, the analytic framework for estimating and interpreting impacts relies on the 
random assignment of participants to treatment and control groups. In theory, random assignment should 
result in treatment and control groups with similar characteristics and allow any observed differences in 
outcomes to be attributable to the intervention. In practice, especially with experimental groups as small 
as those for LLC, treatment and control groups could differ in some characteristics through chance. Large 
differences across experimental groups, however, would raise concerns that the random assignment was 
not executed successfully. To assess the integrity of the random assignment, we examined the baseline 
equivalence of the experimental groups. 


We found that the treatment and control groups were similar across a large set of youth and parent 
characteristics and expectations (Table II.1). Youth characteristics include gender, race, age, grade level, 
types of disabilities, age at disability onset, self-reported health, whether the youth had an active 
vocational rehabilitation case, a 504 plan or an individualized education plan, receipt of Supplemental 
Security Income, eligibility for free or reduced-price school lunch, current or prior employment, and 
expectations for future education and employment. Parent characteristics include education, employment, 
and marital status. We used Student’s t-test to evaluate differences in means between the treatment and 
control groups for binary and continuous variables. For categorical outcomes, we computed 7” statistics to 
determine whether the enrollee distribution across categories was consistent between the two groups. 
Differences between groups for all variables were small and statistically insignificant. 


C. Estimating impacts and regression-adjusted differences 


We use an intent-to-treat framework to estimate LLC’s impacts. With intent to treat, the evaluation 
assesses impacts across all members of the treatment group regardless of their level of engagement with 
services. It does not estimate the impact of the actual program services on the youth that used them 
(whether any individual service or all of them as a package). Instead, it provides inference of what impact 
the program would have if replicated elsewhere in real world settings, with some youth using more 
services than others. We estimated all impact models in Stata 15.1. 


1. Main analysis models 


We used multivariate linear regression models with heteroskedastic-robust standard errors to estimate the 
impacts of LLC. Regression-adjusted estimates are more precise than unadjusted estimates because they 
account for variation across exogenous characteristics. With more precision, we can detect smaller 
program impacts. The general model specification is 


y, =OT, +BX,+e, (1) 
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where y; is the outcome for participant i, 7; is an indicator that equals one for participants in the treatment 
group and zero for participants in the control group, X; is a vector of baseline characteristics, and ¢; is a 
random error term. The parameter 6 represents the impact of LLC on outcome y. 


Given baseline equivalence across characteristics and the relatively small sample size, we limited the 
vector X to characteristics that (1) we were most concerned could be correlated with program outcomes 
and (2) had sufficient variation. These include male gender (compared with female, non-binary, or other), 
non-White or Hispanic race or ethnicity, being in the 9th or 10th grade of high school at enrollment 
(compared with 11th or 12th), having a parent with a college degree, several non-mutually exclusive 
disability categories (attention-deficit/hyperactivity disorder, autism, emotional disorder, intellectual 
disability, learning disability, sensory disability), age at disability onset, and whether the youth received 
income from Supplemental Security Income. If these characteristics are indeed correlated with program 
outcomes, including them as covariates in the regression models should improve the precision of the 
estimates, allowing us to detect smaller impacts. 


2. Weights for survey and earnings outcomes 


We constructed and used weights to adjust estimates for nonresponse. Although we had data for all 
participants in the baseline survey, Division of Vocational Rehabilitation administrative data, and 
National Student Clearinghouse data, we did not have Vermont unemployment insurance wage data, 
youth follow-up survey data, or parent follow-up survey data for subsets of participants. We were able to 
obtain Vermont unemployment insurance wage data only for the 638 participants who provided their 
Social Security number (SSN) at random assignment, and only 378 youth and 374 parents completed the 
follow-up survey. We constructed three weights to adjust for this using propensity score methods (Kalton 
and Flores-Cervantes 2003). We used these weights when estimating impacts for survey-based or 
earnings outcomes to help ensure that the impact estimates reflect the entire analysis sample, not just 
survey respondents or those who provided their SSN. 


3. Accounting for missing data 


We addressed missing data based on how we used the data in the analysis and why the data were missing. 
If data were missing for a baseline characteristic used in the impact analysis as an explanatory variable, 
we used mean imputation to fill in the missing value. For outcome variables, we excluded from impact 
estimation participants with a missing value for the outcome. 


D. Subgroup analyses 


To understand how the impacts of LLC varied across key individual characteristics, we estimated impacts 
for selected subgroups. The subgroups are gender, impairment (has attention-deficit/hyperactivity 
disorder versus does not have it), grade level in high school at enrollment (9th or 10th versus 11th or 
12th), and income-based eligibility for free or reduced-price school lunch. For each subgroup, the 
categories are mutually exclusive. We estimated subgroup impacts for the primary service and 
employment outcomes. 


We chose these subgroups for two reasons. First, because each subgroup reflects characteristics that might 
influence how participants respond to the intervention, program impacts could vary substantively by 
subgroup. Estimating impacts for these subgroups enabled us to explore that variation. Second, the 
analysis sample was well partitioned across the subgroups being compared. The smaller a subgroup is, the 
more difficult it is to detect subgroup impacts. Because of the limited size of the LLC analysis sample, 
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detecting impacts for a small subgroup is especially difficult. By selecting subgroups with more balanced 
representation, we improved the probability of detecting subgroup impacts if any existed. 


We estimated subgroup impacts using a modified version of the main regression model: 


Vi = VS; + oT; + OSyiT; + BX; + Ej (2) 


where Sy is an indicator for being in subgroup y that measures differences in outcome y between 
participants in that subgroup and those not in it. The parameter 6 is the estimated impact of LLC on 
participants not in subgroup y, and 6 is the additional impact of LLC for participants in subgroup y. The 
estimated impact of LLC on people in subgroup y is the sum of 6 and @. To determine whether impacts 
vary across subgroups, we tested whether @ is statistically different from zero. The vector _X includes the 
same covariates as the main regression model. 


E. Presentation of impact estimates 


We report several statistics with each impact estimate. To help readers contextualize the estimates, we 
report the outcome mean for the treatment and control group. We calculated the control group mean 
directly from the data, whereas we regression adjusted the treatment group mean. The impact estimate is 
the difference between the treatment and control group outcome means. We report each impact estimate 
with a heteroskedasticity-robust standard error that indicates the precision of the estimate. To identify 
significant impact estimates, we tested whether each one is statistically different from zero. We report 
whether the impact estimate’s p-value was below key thresholds and highlight in the text and tables those 
impacts with p-values below 10 percent. 


In Appendix D, we also present simple impact estimates—differences in treatment and control group 
means—for the three primary outcomes. Because of the demonstration’s random assignment design, 
differences in mean outcomes across experimental groups are valid impact estimates. Despite being less 
precise than the main regression-adjusted impact estimates, simple impact estimates can reveal whether 
the main measures of LLC’s efficacy are sensitive to regression model selection. 


F. Multiple comparisons 


Simultaneously estimating impacts on multiple outcomes increases the Type 1 error rate (that is, the risk 
of falsely finding impacts when the program did not truly have an impact). Though methods to adjust 
estimates for this exist, these methods can increase the Type 2 error rate (that is, the risk of falsely not 
finding impacts when the program truly did have an impact) (Schochet 2008). With the unavoidable 
challenge of detecting program impacts using the somewhat small LLC sample, we avoided the multiple 
comparisons tradeoff by selecting only one primary outcome within each of the three outcome domains 
that were key LLC components that applied to all treatment group members: 


e Services: at least two WBLEs, including one paid, within 24 months of enrollment 
e Education: enrolled in at least one postsecondary education course by December 2020 


e Employment: had at least one quarter of earnings within the eight quarters after enrollment 


All non-primary outcomes are supplementary outcomes. Though important to understanding the effects of 
the intervention, the supplementary outcomes do not constitute the main assessment of LLC’s impact. 
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G. What Works Clearinghouse evaluation design rating 


An important objective of the LLC evaluation design was to provide evidence of the LLC model that met 
the highest What Works Clearinghouse (WWC) evaluation design rating (that is, the study design meets 
WWC standards without reservations). Achieving this rating is a strong indicator of a rigorous evaluation 
design and thus provides confidence in the validity of the findings. To receive the highest design rating, 
an evaluation must (1) be a randomized controlled trial, (2) have low levels of sample attrition, and (3) 
have at least one outcome that meets criteria for a quality measure (Institute for Educational Studies 
2020). 


The LLC evaluation meets WWC standards for a randomized controlled trial. The WWC standards 
require that a randomized controlled trial assigns each study enrollee to a treatment or control group 
entirely by chance with a nonzero probability. For LLC, we randomly assigned each enrollee with equal 
probability to the treatment group or the control group using an algorithm within our modular, web-based 
management information system known as RAPTER™. The random assignment algorithm in RAPTER® 
ensures randomizing study enrollees properly. Furthermore, Table II.1 reveals strong baseline covariate 
balance between treatment and control group members after random assignment. Though not definitive 
proof, the baseline covariate balance between treatment and control groups is evidence that we conducted 
random assignment correctly. Between understanding the algorithm responsible for random assignment 
and observing strong baseline covariate balance, we are confident in the integrity of random assignment. 


Table A.1. Comparison of overall and differential attrition against WWC standards, for measuring 
primary employment outcome 


ed Red VA 0 (=Ye| led KeDVA (=e | WWC attrition 
Participants SSN (N) SSN (%) Attrition (%) ale -darel(o WL) 


Total 803 638 79.4 20.5 55.0 
Treatment group 413 339 82.1 17.9 
Control group 390 299 76.7 23.3 
Differential attrition 5.4 5.4 


SSN = Social Security number; WWC = What Works Clearinghouse. 


We must also meet the WWC standards for attrition. Attrition occurs when the evaluation lacks an 
outcome variable for a study enrollee. Certain levels of attrition can compromise the comparability of the 
treatment and control groups, potentially biasing estimates of the intervention’s outcomes. WWC 
standards examine two types of attrition: overall attrition—the rate of attrition for the entire sample—and 
differential attrition—the difference in rates of attrition between the treatment and control groups. Both 
types of attrition must be low to achieve the highest WWC design rating, which the standards refer to as 
“tolerable levels of potential bias for both the optimistic and cautious sets of assumptions.” 


When we examine overall and differential attrition, we find that the data we used to measure the three 
primary outcomes meet WWC’s criteria for low overall and differential sample attrition. For the service 
outcome measured using Division of Vocational Rehabilitation administrative data and the postsecondary 
outcome measured using National Student Clearinghouse data, attrition is a non-issue because we have 
data on all participants. Measurement of the primary employment outcome, however, was limited to those 
participants who provided a valid SSN. The 638 participants that did provide SSN represent 79 percent of 
the sample for an overall sample attrition rate of 21 percent. The rate was slightly higher among the 
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treatment group (82 percent) than the control group (77 percent), for a differential attrition rate of 5.4 
percent. The combination of the 21 percent sample attrition rate and the 5.4 differential attrition rate are 
within the WWC’s cautious boundary for attrition. 


To receive the highest rating, WWC standards also require that at least one outcome meets review 
requirements and is free of confounding factors. Both the primary service and education outcomes meet 
this review requirement. These measures have face validity because they are clearly defined, and the 
content they assess aligns with their definitions. Because the measures are recorded in the same manner 
for both experimental groups, and because we observe strong baseline covariate balance across groups, 
we are confident that no component of the study design or feature of implementation for these measures is 
perfectly aligned or confounded with either experimental group. 
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Table B.1. Differential impacts of LLC on receipt of two WBLEs, including one paid, by personal 
characteristics and timing of enrollment 


Impact 
Treatment Control estimate | Treatment 

YU] ote] cole] 9) mean mean (T vs. C) Control N 
Disability type 
Does not have ADHD 25.0 13.4 11.6*** (3.8) 225 202 
Has ADHD 24.9 4.8 20.1*** (3.5) 188 188 
Differential treatment impact 8.5 (5.2) 
Enrollment cohort 
Early 23.9 12.2 11.7*** (4.1) 174 164 
Late 25.5 7.1 18.4*** (3.3) 239 226 
Differential treatment impact -6.7 (5.2) 
10 22.3 7.1 15.2*** (3.6) 189 182 
11 27.1 11.1 16.0*** (3.7) 224 208 
Differential treatment impact -0.1 (5.2) 
Male 23.1 6.5 16.6*** (3.1) 254 232 
Not male 27.3 13.3 14.0*** (4.4) 159 158 
Differential treatment impact 2.6 (5.4) 
eligibility 
Eligible 24.8 6.3 18.5*** (3.4) 214 205 
Not eligible 24.9 12.4 12.5*** (4.0) 199 185 
Differential treatment impact 6.0 (5.3) 


Source: DVR administrative data and LLC baseline survey. 


Notes: We regression adjusted treatment means and differences to account for baseline characteristics. Standard 
errors of the impact estimates are in parentheses. 


*/**/*** Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 
Significance thresholds are not adjusted for multiple comparisons. 


ADHD = attention-deficit/nyperactivity disorder; DVR = Division of Vocational Rehabilitation; LLC = Linking Learning 
to Careers; WBLE = work-based learning experience. 


Mathematica 


45 


Appendix B Additional Impact and Supplementary Outcomes Estimates 


Table B.2. Differential impacts of LLC on having at least one quarter with earnings, by personal 
characteristics and timing of enrollment 


Impact 
Treatment OFoJ al i Ke)| estimate | Treatment 

YU] ote] ole] 9) mean mean (T vs. C) Control N 
Disability type 
Does not have ADHD 62.6 61.4 1.2 (5.4) 183 152 
Has ADHD 69.0 60.5 8.5 (5.3) 156 147 
Differential treatment impact 7.3 (8.0) 
Early 65.2 70.2 -5.0 (5.8) 142 120 
Late 66.0 54.8 11.2** (4.9) 197 179 
Differential treatment impact -16.2 (8.0)** 
10 61.2 52.0 9.2 (5.8) 155 133 
11 69.0 68.1 1.0 (5.0) 184 166 
Differential treatment impact 8.2 (7.6) 
[eT-YaTe (= 
Male 66.2 61.2 5.1 (4.9) 206 177 
Not male 64.7 60.7 4.1 (5.9) 133 122 
Differential treatment impact 1.0 (7.8) 
at =X: Wo) am c-Yo[ULex-Ye ts olalex-M-Yelavole) MU later 
eligibility 
Eligible 63.4 59.4 4.1 (5.3) 178 160 
Not eligible 68.2 62.8 5.3 (5.3) 161 139 
Differential treatment impact 1.3 (7.5) 


Source: Vermont wage records and LLC baseline survey. 


Notes: We regression adjusted treatment means and differences to account for baseline characteristics. Standard 
errors of the impact estimates are in parentheses. 


*/*/*** Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 
Significance thresholds are not adjusted for multiple comparisons. 


ADHD = attention-deficit/hyperactivity disorder; LLC = Linking Learning to Careers. 
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ee TT 
Table B.3. Impacts of LLC on primary service and employment outcomes within 24 months of 


enrollment, by district office at enrollment 


District office 


Barre-Montpelier 


Treatment 


Impact 
estimate 
(T vs. C) 


(OFeyaii ne) | 


mean mean 


Services: Had at least two WBLEs, including 28.5 2.7 25.8*** (8.3) 36 37 
one paid 
Employment: Had at least one quarter of 79.0 72.4 6.6 (13.6) 29 29 


earnings 


Services: Had at least two WBLEs, including 9.9 5.4 4.5 (5.8) 38 37 
one paid 
Employment: Had at least one quarter of 53.3 61.1 -7.9 (12.0) 38 36 


earnings 


= Yer-}aa (=) ole) ge) 


Services: Had at least two WBLEs, including 
one paid 

Employment: Had at least one quarter of 
earnings 


29.3*** (9.7) 39 34 


15.4 (13.1) 36 29 


Burlington 


Services: Had at least two WBLEs, including 21.1 4.2 16.9** (6.6) 49 48 
one paid 
Employment: Had at least one quarter of 17.1 63.0 14.1 (9.0) 47 46 


earnings 


Middlebury 


Services: Had at least two WBLEs, including 44.3 0.0 44.3*** (12.1) 17 12 
one paid 
Employment: Had at least one quarter of 86.9 91.7 -4.8 (12.4) 17 12 


earnings 


Morrisville 


Services: Had at least two WBLEs, including 27.1 9.1 18.0* (9.9) 34 33 
one paid 
Employment: Had at least one quarter of 64.0 51.6 12.4 (11.6) 32 31 


earnings 


Services: Had at least two WBLEs, including 53.4 20.6 32.8*** (11.6) 35 34 
one paid 
Employment: Had at least one quarter of 69.4 63.2 6.2 (13.0) 27 19 


earnings 


Services: Had at least two WBLEs, including 
one paid 

Employment: Had at least one quarter of 
earnings 


3.7 (9.9) 34 33 


-18.1 (26.3) 17 16 
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District office 


Springfield 


Impact 
estimate 
(T vs. C) 


(eFeyaii Ke) | 
mean 


Treatment 
mean 


Services: Had at least two WBLEs, including 18.8 3.7 15.1* (8.3) 29 27 
one paid 
Employment: Had at least one quarter of 79.8 35.3 44.5** (17.5) 15 17 


earnings 


St. Albans 


Services: Had at least two WBLEs, including 26.0 29.0 -3.0 (13.5) 34 31 
one paid 
Employment: Had at least one quarter of 67.6 74.2 -6.6 (12.8) 34 31 


earnings 


St. Johnsbury 


Services: Had at least two WBLEs, including 30.4 16.1 14.3 (12.7) 34 31 
one paid 

Employment: Had at least one quarter of 60.0 60.0 0.0 (22.9) 25 15 
earnings 

White River Junction 

Services: Had at least two WBLEs, including 6.3 0.0 6.3 (7.1) 34 33 
one paid 

Employment: Had at least one quarter of 46.5 50.0 -3.5 (19.7) 22 18 
earnings 

Source: DVR administrative data and Vermont unemployment insurance data. 


Notes: 


We regression adjusted treatment means and differences to account for baseline characteristics. Standard 


errors of the impact estimates are in parentheses. 
*/**/*** Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 
Significance thresholds are not adjusted for multiple comparisons. 
DVR = Division of Vocational Rehabilitation; LLC = Linking Learning to Careers; WBLE = work-based learning 


experience. 
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a ee 
Table B.4. Sensitivity of estimates for primary outcomes to sample restrictions 


Treatment 
Outcome mean Control mean| Difference 
Number of students in demonstration 
Number that completed 24-month survey 214 164 
Number in early cohort 174 164 
Number in late cohort 239 197 
Services: Had at least two (one CIE) WBLEs since 24.9 9.3 15.6*** (2.6) 
enrollment (all) 
Students that completed 24-month survey 27.0 13.4 13.6*** (4.1) 
Early cohort 23.9 12.2 11.7*** (4.1) 
Late cohort 25.5 7.1 18.4*** (3.3) 
Education: Ever enrolled in postsecondary education (all) 30.6 22.4 8.2*** (3.1) 
Early cohort 34.5 25.5 9.0*** (4.1) 
Late cohort 24.5 17.6 6.9 (4.6) 
Employment: Worked in paid employment (all) 65.6 61.0 4.7 (3.8) 
Students that completed 24-month survey 63.3 63.3 0.0 (5.0) 
Early cohort 65.2 70.2 -5.0 (5.8) 
Late cohort 66.0 54.8 11.2** (4.9) 
Source: DVR administrative data, NSC data, and Vermont unemployment insurance data. 
Note: We regression adjusted treatment means and differences to account for baseline characteristics. Standard 


errors of the impact estimates are in parentheses. 


*r“"* Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 
Significance thresholds are not adjusted for multiple comparisons. 


CIE = competitive integrated employment; DVR = Division of Vocational Rehabilitation; NSC = National Student 
Clearinghouse; WBLE = work-based learning experience. 
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Table B.5. Other service outcomes within 24 months of enrollment 


Treatment mean | Control mean Difference 

Number of respondents 214 164 
DVR 44.1 25.7 18.4*** (4.9) 
Different agency serving people with 12.4 9.9 2.4 (3.1) 
disabilities 
An American Job Center 0.7 1.2 -0.4 (0.9) 
A high school or other secondary school 40.8 48.1 -7.3 (5.2) 
A postsecondary school 8.8 5.1 3.7 (2.7) 
Some other place 6.9 3.4 3.5 (2.4) 
Took tours of workplaces 39.3 33.2 6.1 (4.9) 
Participated in a job shadow 36.2 44.1 -8.0 (5.1) 
Interviewed someone about their job 24.3 19.2 5.1 (4.3) 
Regularly talked one on one with someone 42.8 50.7 -7.9 (5.1) 
about jobs 
Regularly talked as part of a group with 30.3 30.3 0.0 (4.8) 
someone from outside school about jobs 
Had a paid internship or apprenticeship 12.3 16.4 -4.1 (3.7) 
Had an unpaid internship or apprenticeship 16.1 13.6 2.4 (3.7) 
Worked in a school-based enterprise 19.4 26.9 -7.4 (4.4) 

MY, ol:\-Wo) 1-1 avd (1-¥-M al-V-Le (Vem oLUi mate) Mac\er-) \(-Le| 
Discovering job interests and job skills 4.5 4.1 0.4 (2.1) 
Career counseling 2.6 4.0 -1.5 (1.8) 
Learning how to look for a job 4.1 3.8 0.3 (2.1) 
Job shadowing 3.3 3.5 -0.2 (1.9) 
An apprenticeship or internship 1.7 3.0 -1.3 (1.6) 
Help finding a job 6.2 4.6 1.6 (2.4) 
Support on the jobs, such as job coaching 2.7 3.9 -1.2 (1.9) 
Help getting into school or training 2.0 3.7 -1.8 (1.8) 
Understanding SSA or other benefits 1.5 2.3 -0.7 (1.5) 
Computer literacy classes 1.6 1.7 -0.1 (1.4) 
Problem solving 4.2 4.8 -0.6 (2.2) 
Financial literacy training 1.7 2.9 -1.2 (1.5) 
Referral to another agency 0.5 1.8 -1.4 (1.1) 
Transportation services 5.0 1.9 3.1 (1.9) 
Health-related services 0.5 1.5 -1.0 (1.0) 
Case management 1.0 1.5 -0.5 (1.1) 
Other services 0.6 2.5 -1.9 (1.3) 
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AYE-Var-Ve) (=) Treatment mean | Control mean Difference 
AT 
Always or often use AT tools 26.6 21.6 5.0 (4.4) 
Very satisfied with AT tools 53.7 31.0 22.6** (8.8) 
Received assistance with AT 57.1 44.5 12.6 (9.2) 
Very satisfied with AT assistance> 47.6 57.5 -9.9 (14.0) 


Source: LLC youth 24-month follow-up survey. 


Notes: We regression adjusted treatment means and differences to account for baseline characteristics. Standard 
errors of the impact estimates are in parentheses. 


*“r“/"* Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 
Significance thresholds are not adjusted for multiple comparisons. 


4 We measured satisfaction and estimated means only among those that use AT tools. 
> We measured satisfaction and estimated means only among those that received assistance with AT tools. 


AT = assistive technology; DVR = Division of Vocational Rehabilitation; LLC = Linking Learning to Careers; SSA = 
Social Security Administration. 
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Table B.6. Other education and training outcomes within 24 months of enrollment 


Variable 
Number of youth respondents 


BBY/ol-Moymlarieiieiacoyam-Valce)|(-Xemlam-lmtiii(- Me) mialcyavd(c\i 


(Oey al ine) | 
mean 


Treatment 


mean Difference 


High school 61.0 68.3 -7.3 (5.3) 
High school that serves only participants with disabilities 2.3 3.3 -1.0 (1.8) 
School that provides occupational or technical training, usually 9.6 9.0 0.6 (3.8) 
less than two years 

Two-year community college 9.8 5.4 4.5 (2.8) 
Four-year college 11.1 9.0 2.1 (3.4) 
Another type of school 5.5 5.1 0.4 (2.5) 


bale] at=soime(-Xe]x-1-Me) mm (=)14-) Moy Keg aLole) Mexe) aay e)(=1c-Le| 


Some high school, no diploma 37.0 40.1 -3.2 (4.4) 
High school with diploma or equivalent 46.1 52.9 -6.8 (4.9) 
Certificate of completion 1.2 1.8 -0.6 (1.1) 
Trade, technical, or vocational training 2.5 1.8 0.6 (1.6) 
Associate degree 0.0 0.0 -- 
Bachelor’s degree or higher 0.0 0.0 -- 
Any other type of degree or schooling 2.3 0.0 2.3* (1.2) 
Some college, no diploma 10.9 3.4 7.5*** (2.8) 
In the past year, youth learned the following leadership 
activities 
Working in a team 60.5 59.3 1.2 (5.7) 
Making decisions 70.3 68.7 1.5 (5.4) 
Handling conflict 62.9 63.0 -0.1 (5.4) 
Number of parent respondents 214 160 
Youth received special education services or had an 83.1 77.9 5.2 (4.3) 
individualized education plan 
Youth had a section 504 plan 9.9 13.3 -3.4 (3.5) 
Youth had a transition planning meeting to help plan what they 78.1 76.5 1.6 (5.0) 


might do after high school 


MU it-MYColel iam ae) (-Miamee-lavitcolam delete) 


Provided some input 


16.1*** (5.3) 


Took a leadership role 


At the transition planning meeting 


-8.4* (4.8) 


The youth’s interests, strengths, and preferences were 75.2 72.0 3.2 (5.1) 
discussed 
Staff from a community service agency took part 47.5 33.6 13.9** (5.5) 
The youth received information on education, careers, or 57.2 48.9 8.3 (5.5) 
community living options for when they leave high school 

Parent or guardian helped the youth prepare for life after 

balke]amXeatexe) Mdalcel¥ie]aMmant-mcodl re iUlare M-Ceut\ ai i(sc) 
Attended a career day or job fair 34.3 31.8 2.5 (5.7) 
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Treatment (Oey al ine) | 
Variable mean mean Difference 


Arranged to attend a program or take a tour of a college 51.7 52.3 -0.6 (5.7) 
campus 

Arranged to sit in on or take a college class 40.2 29.7 10.5** (5.3) 
Arranged to participate in an internship or apprenticeship 37.0 37.7 -0.7 (5.8) 
related to career goals 

Arranged to work or volunteer in a job related to career goals 50.1 52.7 -2.6 (5.8) 
Searched the Internet for college options or read college 44.8 50.8 -6.0 (5.9) 
guides 

Talked with a high school counselor about their options for life 68.6 60.1 8.5 (5.5) 


after high school 


Talked about options with a counselor hired by family to help 11.1 10.1 0.9 (3.3) 
the participant prepare for college admission 


Arranged to take a course to prepare for a college admission 24.0 14.8 9.2** (4.4) 
exam such as the SAT or ACT 


Source: LLC youth and parent 24-month follow-up surveys. 


Notes: We regression adjusted treatment means and differences to account for baseline characteristics. Standard 
errors of the impact estimates are in parentheses. 


*/**/*** Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 
Significance thresholds are not adjusted for multiple comparisons. 


LLC = Linking Learning to Careers. 
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Table B.7. Other employment outcomes within 24 months of enrollment 


Treatment mean Control mean Difference 
Number of respondents 214 164 
Employed in past year 79.2 83.8 -4.6 (4.0) 
Number of jobs in the past year 1.4 1.4 0.0 (0.1) 
[characteristics of most recent 
Job was paid 87.9 95.3 -7.4** (3.2) 
Employed at time of survey 62.1 63.5 -1.4 (5.5) 
Part of a school-sponsored activity 11.9 15.7 -3.8 (4.0) 
Youth very or somewhat satisfied 89.7 83.7 6.0 (3.8) 
Wage for those paid hourly ($) 12.0 11.9 0.1 (0.3) 
Benefits 
Health insurance 13.1 5.5 7.6** (3.3) 
Paid vacation, holidays, or sick days 13.6 12.4 1.2 (3.9) 
Retirement 8.1 7.3 0.8 (3.1) 
Supports received 
Work clothes or uniforms 43.8 51.2 -7.5 (5.8) 
Work-related equipment 16.3 24.2 -7.8* (4.6) 
Transportation 23.8 29.4 -5.6 (4.8) 
Assistive technology 7.5 7.7 -0.3 (3.0) 
Accommodations 43.0 47.3 -4.3 (5.8) 
Other supports 7.0 9.7 -2.7 (3.0) 
How youth found job 
Vermont Division of Vocational 12.0 7.1 4.8 (3.5) 
Rehabilitation 
Friends, relatives, or colleagues 38.8 43.8 -5.0 (5.5) 
School 15.7 18.3 -2.6 (4.3) 
Other 31.2 30.1 1.1 (5.3) 
Number of participants with SSN 339 299 
Any earnings across 8 quarters 65.6 61.0 4.7 (3.8) 
1st quarter after enrollment 28.6 27.9 0.6 (3.6) 
2nd quarter after enrollment 32.1 30.1 2.1 (3.7) 
3rd quarter after enrollment 36.0 35.7 0.3 (3.8) 
4th quarter after enrollment 36.2 38.5 -2.3 (3.9) 
5th quarter after enrollment 39.1 40.9 -1.8 (3.9) 
6th quarter after enrollment 40.3 39.7 0.6 (3.9) 
7th quarter after enrollment 45.2 43.0 2.2 (3.9) 
8th quarter after enrollment 46.8 42.9 3.9 (4.0) 
Earnings amount across 8 quarters ($) 6,258.3 6,774.5 -516.2 (717.2) 
1st quarter after enrollment ($) 369.1 428.8 -59.7 (69.1) 
2nd quarter after enrollment ($) 437.6 527.0 -89.4 (86.7) 


Mathematica 


54 


Appendix B Additional Impact and Supplementary Outcomes Estimates 


Variable Treatment mean Control mean Difference 


3rd quarter after enrollment ($) 661.2 749.9 -88.7 (107.2) 
4th quarter after enrollment ($) 644.7 688.8 -44.1 (100.9) 
5th quarter after enrollment ($) 710.3 787.0 -76.6 (102.9) 
6th quarter after enrollment ($) 940.6 1,013.7 -73.1 (148.2) 
7th quarter after enrollment ($) 1,142.1 1,260.0 -117.9 (155.1) 
8th quarter after enrollment ($) 1,352.6 1,319.3 33.3 (169.0) 
Fany earnings by calendar quarter ta 

Q1 2019 31.0 36.5 -5.5 (3.7) 
Q2 2019 36.4 39.7 -3.3 (3.8) 
Q3 2019 45.8 43.5 2.3 (3.8) 
Q4 2019 42.8 40.9 1.8 (3.8) 
Q1 2020 44.6 42.1 2.5 (3.8) 
Q2 2020 40.1 35.0 5.1 (3.7) 
Q3 2020 45.7 42.6 3.1 (3.8) 
Q4 2020 44.7 44.3 0.4 (3.9) 
Q1 2019 ($) 494 591 -96.7 (86.6) 
Q2 2019 ($) 555 665 -109.8 (90.1) 
Q3 2019 ($) 1,045 1,204 -159.5 (144.4) 
Q4 2019 ($) 1,021 1,066 -45.4 (144.0) 
Q1 2020 ($) 1,011 1,052 -41.2 (144.4) 
Q2 2020 ($) 988 1,090 -102.4 (154.9) 
Q3 2020 ($) 1,592 1,605 -13.3 (191.6) 
Q4 2020 ($) 1,954 1,734 219.8 (230.6) 


Source: LLC youth 24-month follow-up surveys and Vermont unemployment insurance data. 


Notes: We regression adjusted treatment means and differences to account for baseline characteristics. Standard 
errors of the impact estimates are in parentheses. 


*P*/*** Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 
Significance thresholds are not adjusted for multiple comparisons. 


LLC = Linking Learning to Careers; Q = quarter; SSN = Social Security number. 
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_————— oe 
Table B.8. Expectations for the future, reported 24 months after enrollment 


Treatment 
Variable mean Control mean| Difference 


Number of youth respondents 214 164 

Expectations for living situation at age 30 
With a parent or guardian 9.3 10.4 -1.2 (3.4) 
With a sibling or other relative 3.3 2.7 0.6 (1.8) 
On their own or with spouse or partner 78.0 79.2 -1.2 (4.5) 
In a group home or institution 0.1 1.5 -1.3 (1.0) 
In another living situation 3.5 1.9 1.6 (1.8) 

Number of parent respondents 214 160 

Parent expects youth to definitely or probably earn enough to 66.0 68.3 -2.3 (4.9) 


support self without financial help from family or government 
benefit programs 


Parent expectations for youth’s living situation at age 30 


With a parent or guardian 16.9 13.6 3.2 (4.1) 
With a sibling or other relative 1.9 4.2 -2.3 (1.7) 
On their own or with spouse or partner 62.5 62.1 0.3 (5.3) 
In a group home or institution 2.2 2.6 -0.3 (2.2) 
In another living situation 8.6 10.4 -1.8 (3.1) 


Source: LLC youth and parent 24-month follow-up surveys. 


Notes: We regression adjusted treatment means and differences to account for baseline characteristics. Standard 
errors of the impact estimates are in parentheses. 


*/**/*** Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 
Significance thresholds are not adjusted for multiple comparisons. 


LLC = Linking Learning to Careers. 


Mathematica 


Appendix C 


Unadjusted Impacts of LLC on Primary Outcomes 


Mathematica 


This page has been left blank for double-sided copying. 


58 


Appendix C Unadjusted Impacts of LLC on Primary Outcomes 


Table C.1. Unadjusted differences in primary outcomes between treatment and control groups 


within 24 months of enrollment 


OTT] Kereta) Treatment mean | Control mean Difference 
Number of participants 413 390 

Services: Had at least two WBLEs, including one paid 24.9 9.2 15.7*** (2.6) 
Education: Ever enrolled in postsecondary education 30.6 22.4 8.2*** (3.1) 
Number of participants with SSN 339 299 

Employment: Had at least one quarter of earnings 66.0 61.0 5.0 (3.9) 


Source: DVR administrative data and Vermont unemployment insurance data. 


Notes: We regression adjusted treatment means and differences to account for baseline characteristics. Standard 


errors of the impact estimates are in parentheses. 


“r“r"* — Difference is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test. 


Significance thresholds are not adjusted for multiple comparisons. 


DVR = Division of Vocational Rehabilitation; SSN = Social Security number; WBLE = work-based learning 


experience. 
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